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Abstract 


Randomization is an exceptional tool for the design of distributed algorithms, sometimes yield- 
ing efficient solutions to problems that are inherently complex, or even unsolvable, in the setting 
of deterministic algorithms. However, this tool has a price: even simple randomized algorithms 
can be extremely hard to verify and analyze. 

This thesis addresses the problem of verification of randomized distributed algorithms. We 
consider the problem both from the theoretical and the practical perspective. Our theoretical 
work builds a new mathematical model of randomized distributed computation; our practical 
work develops techniques to be used for the actual verification of randomized systems. Our 
analysis involves both untimed and timed systems, so that real-time properties can be investi- 
gated. 

Our model for randomized distributed computation is an extension of labeled transition 
systems. A probabilistic automaton is a state machine with transitions, where, unlike for labeled 
transition systems, a transition from a state leads to a discrete probability distribution over pairs 
consisting of a label and a state, rather than to a single label and a single state. A probabilistic 
automaton contains pure nondeterministic behavior since from each state there can be several 
transitions, and probabilistic behavior since once a transition is chosen the label that occurs and 
the state that is reached are determined by a probability distribution. The resolution of pure 
nondeterminism leads to probabilistic executions, which are Markov chain like structures. Once 
the pure nondeterminism is resolved, the probabilistic behavior of a probabilistic automaton 
can be studied. 

The properties of a randomized algorithm are stated in terms of satisfying some other prop- 
erty with a minimal or maximal probability no matter how the nondeterminism is resolved. 
In stating the properties of an algorithm we also account for the possibility of imposing re- 
strictions on the ways in which the nondeterminism is resolved (e.g., fair scheduling, oblivious 
scheduling,...). We develop techniques to prove the correctness of some property by reducing 
the problem to the verification of properties of non-randomized systems. One technique is 
based on coin lemmas, which state lower bounds on the probability that some chosen random 
draws give some chosen outcomes no matter how the nondeterminism is resolved. We identify 
a collection of progress statements which can be used to prove upper bounds to the expected 
running time of an algorithm. The methods are applied to prove that the randomized dining 
philosophers algorithm of Lehmann and Rabin guarantees progress in expected constant time 
and that the randomized algorithm for agreement of Ben-Or guarantees agreement in expected 
exponential time. 

To ensure that our new model has strong mathematical foundations, we extend some of the 


common semantics for labeled transition systems to the probabilistic framework. We define a 
compositional trace semantics where a trace is replaced by a probability distribution over traces, 
called a trace distribution, and we extend the classical bisimulation and simulation relations in 
both their strong and weak version. Furthermore, we define probabilistic forward simulations, 
where a state is related to a probability distribution over states. All the simulation relations 
are shown to be sound for the trace distribution semantics. 

In summary, we obtain a framework that accounts for the classical theoretical results of 
concurrent systems and that at the same time proves to be suitable for the actual verification 
of randomized distributed real-time systems. This double feature should lead eventually to the 
easy extension of several verification techniques that are currently available for non-randomized 
distributed systems, thus rendering the analysis of randomized systems easier and more reliable. 
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Chapter 1 


Introduction 


1.1 The Challenge of Randomization 


In 1976 Rabin published a paper titled Probabilistic Algorithms [Rab76] where he presented 
efficient algorithms for two well-known problems: Nearest Neighbors, a problem in computa- 
tional geometry, and Primality Testing, the problem of determining whether a number is prime. 
The surprising aspect of Rabin’s paper was that the algorithms for Nearest Neighbors and for 
Primality Testing were efficient, and the key insight was the use of randomized algorithms, 
i.e., algorithms that can flip fair coins. Rabin’s paper was the beginning of a new trend of 
research aimed at using randomization to improve the complexity of existing algorithms. It is 
currently conjectured that there are no efficient deterministic algorithms for Nearest Neighbors 
and Primality Testing. 

Another considerable achievement came in 1982, when Rabin [Rab82] proposed a solution 
to a problem in distributed computing which was known to be unsolvable without random- 
ization. Specifically, Rabin proposed a randomized distributed algorithm for mutual exclusion 
between n processes that guarantees no-lockout (some process eventually gets to the critical 
region whenever some process tries to get to the critical region) and uses a test-and-set shared 
variable with O(log) values. On the other hand, Burns, Fisher, Jackson, Lynch and Patter- 
son [BFJ*82] showed that Q(n) values are necessary for a deterministic distributed algorithm. 
Since then, several other randomized distributed algorithms were proposed in the literature, 
each one breaking impossibility results proved for deterministic distributed algorithms. Several 
surveys of randomized algorithms are currently available; among those we cite [Kar90, GSB94]. 

The bottom line is that randomization has proved to be exceptionally useful for problems in 
distributed computation, and it is slowly making its way into practical applications. However, 
randomization in distributed computation leaves us with a challenge whose importance increases 
as the complexity of algorithms increases: 


“How can we analyze randomized distributed algorithms? In particular, how can we 
convince ourselves that a randomized distributed algorithm works correctly?” 


The analysis of non-randomized distributed systems is challenging already, due to a phenomenon 
called nondeterminism. Specifically, whenever two systems run concurrently, the relative speeds 
of the two systems are not known in general, and thus it is not possible to establish a priori 
the order in which the systems complete their tasks. On the other hand, the ordering of the 
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completion of different tasks may be fundamental for the global correctness of a system, since, 
for example, a process that completes a task may prevent another process from completing 
its task. The structure of the possible evolutions of a system can become intricate quickly, 
justifying the statement “there is rather a large body of sad experience to indicate that a 
concurrent program can withstand very careful scrutiny without revealing its errors” [OL82]. 

The introduction of randomization makes the problem even more challenging since two 
kinds of nondeterminism arise. We call them pure nondeterminism and probabilistic nondeter- 
minism. Pure nondeterminism is the nondeterminism due to the relative speeds of different 
processes; probabilistic nondeterminism is the nondeterminism due to the result of some ran- 
dom draw. Alternatively, we refer to pure nondeterminism as the nondeterministic behavior of 
a system and to probabilistic nondeterminism as the probabilistic behavior of a system. The 
main difficulty with randomized distributed algorithms is that the interplay between probabil- 
ity and nondeterminism can create subtle and unexpected dependencies between probabilistic 
events; the experience with randomized distributed algorithms shows that “intuition often fails 
to grasp the full intricacy of the algorithm” [PZ86], and “proofs of correctness for probabilistic 
distributed systems are extremely slippery” [LR81]. 

In order to meet the challenge it is necessary to address two main problems. 


e Modeling: How do we represent a randomized distributed system? 
e Verification: Given the model, how do we verify the properties of a system? 


The main objective of this thesis is to make progress towards answering these two questions. 


1.1.1 Modeling 


First of all we need a collection of mathematical objects that describe a randomized algorithm 
and its behavior, i.e., we need a formal model for randomized distributed computation. The 
model needs to be sufficiently expressive to be able to describe the crucial aspects of randomized 
distributed computation. Since the interplay between probability and nondeterminism is one 
of the main sources of problems for the analysis of an algorithm, a first principle guiding our 
theory is the following: 


1. The model should distinguish clearly between probability and nondeterminism. 


That is, if either Alice or Bob is allowed to flip a coin, the choice of who is flipping a coin is 
nondeterministic, while the outcome of the coin flip is probabilistic. 

Since the model is to be used for the actual analysis of algorithms, the model should allow 
the description of randomized systems in a natural way. Thus, our second guiding principle is 
the following: 


2. The model should correspond to our natural intuition of a randomized system. 


That is, mathematical elegance is undoubtedly important, but since part of the verification 
process for an algorithm involves the representation of the algorithm itself within the formal 
model, the chance of making errors is reduced if the model corresponds closely to our view of 
a randomized algorithm. A reasonable tradeoff between theory and practice is necessary. 
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Our main intuition for a computer system, distributed or not, is as a state machine that 
computes by moving from one state to another state. This intuition leads to the idea of Labeled 
Transition Systems (LTS) [Kel76, Plo81]. A labeled transition system is a state machine with 
labels associated with the transitions (the moves from one state to another state). Labeled 
transition systems have been used successfully for the modeling of ordinary distributed systems 
[Mil89, Jon91, LV91, LT87, GSSL94], and for their verification [WLL88, SLL93, SGGT93, 
BPV94]; in this case the labels are used to model communication between several systems. Due 
to the wide use of labeled transition systems, the extensive collection of verification techniques 
available, and the way in which labeled transition systems correspond to our intuition of a 
distributed system, two other guiding principles for the thesis are the following: 


3. The new model should extend labeled transition systems. 


4, The extension of labeled transition systems should be conservative, i.e., whenever a system 
does not contain any random choices, our new system should reduce to an ordinary labeled 
transition system. 


In other words our model is an extension of the labeled transition system model so that ordinary 
non-randomized systems turn out to be a special case of randomized systems. Similarly, all the 
concepts that we define on randomized systems are generalizations of corresponding concepts 
of ordinary non-randomized systems. In this way all the techniques available should generalize 
easily without the need to develop completely new and independent techniques. Throughout 
the thesis we refer to labeled transition systems as automata and to their probabilistic extension 
as probabilistic automata. 


1.1.2 Verification 


Once the model is built, our primary goal is to use the model to describe the properties that 
a generic randomized algorithm should satisfy. If the model is well designed, the properties 
should be easy to state. Then, our second goal is to develop general techniques that can be 
used for verification. 

We investigate verification techniques from two perspectives. On one hand we formalize 
some of the kinds of the informal arguments that usually appear in existing papers; on the 
other hand we extend existing abstract verification techniques for labeled transition systems 
to the probabilistic framework. Examples of abstract techniques include the analysis of traces 
[Hoa85], which are ordered sequences of labels that can occur during the evolution of a system, 
and of simulation relations [Mil89, Jon91, LV91], which are relations between the states of 
two systems such that one system can simulate the transitions of the other via the simulation 
relation. To provide some intuition for traces and simulations, Figure 1-1 represents three 
labeled transition systems, denoted by A,, Ag, and Az. The empty sequence and the sequences 
a and ab are the traces of A,, Ag, and Ag. For example, a computation that leads to ad is the 
one that starts from 59, moves to s,, and then to s3. The dotted lines from one state to another 
state (the arrows identify the from-to property) are examples of simulation relations from one 
automaton to the other. For example, consider the simulation relation from A3 to Ag. State so 
of Ag is related to state 59 of Ag; states s, and s of Ag are related to state sy of A; state sz 
of Az is related to state s3 of Ay. The transition of As from sp to sy with action a is simulated 
in Ay by the transition from sg to 5, with label a. There is a strong simulation also from A» 
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Figure 1-1: Simulation relations for automata. 


to Az (each state s; of Ag is related to state s; of A3), from Ay to Ag, and from Ag to Aj. 
There is an even stronger relation between A, and Ag, which is called a bisimulation and is 
represented by the double-arrow dotted lines between the states of Ay and Ag. A bisimulation 
is an equivalence relation between the states of two automata. In this case each automaton can 
simulate the transitions of the other via the bisimulation relation. 


Direct Verification 


In the description of a randomized distributed algorithm pure nondeterminism represents the 
undetermined part of its behavior, namely, in what order the processes are scheduled. Schedul- 
ing processes is the activity of removing the nondeterminism, and the object that does the 
scheduling is usually referred to as a scheduler or an adversary. The intuition behind the name 
“adversary” is in proving the correctness of an algorithm a scheduler is viewed as a malicious 
entity that degrades the performance of the system as much as possible. 

Once the nondeterminism is removed, a system looks like a Markov chain, and thus it is 
possible to reason about probabilities. A common argument is then 


“no matter how the scheduler acts, the probability that some good property holds is 
at least p.” 


Actually, in most of the existing work p is 1, since the proofs are easier to carry out in this case. 
In this thesis we are interested in every p since we are concerned also with the time complexity 
of an algorithm. Throughout the thesis it will become clear why we need every p for the study 
of time complexity. 

One of our major goals is to remove from the informal arguments of correctness all “danger- 
ous” statements, i.e., all statements that rely solely on intuition rather than on actual deduc- 
tions, and yet keep the structure of a proof simple. In other words, we want to provide tools 
that allow people to argue as before with a significantly higher confidence that what they say is 
correct. Then, we want to develop techniques that allow us to decompose the verification task 
of complex properties into simpler verification tasks. This feature is important for scalability. 
Here we give examples of two issues that we believe to be important. 


e Make sure that you know what probability space you are working in. Or, at least, make 
sure that you are working in a probability space. This is a rule of thumb that is valid in 
other fields like Information Theory and Detection Theory. Probability is very tricky. The 


16 


fact that a specific probability space was not identified was the reason for a bug discovered 
by Saias [Sai92] in the original algorithm of Rabin [Rab82], later fixed by Kushilevitz and 
Rabin [KR92]. Of course, in order to make sure we know what probability spaces we are 
working in, we need some easy mechanisms to identify those probability spaces. Such 
mechanisms were not available in 1982. 


e Avoid arguments of the kind “now the worst thing that can happen is the following.” 
These arguments are usually based on the intuition that the designers have about their 
own algorithm. Specifically, as has happened in the past, the designers argue based on 
worst cases they can think of rather than the actual worst case. What is missing is a 
proof showing that the worst case has been identified. A much better statement would 
be “no matter what happens, something else will happen”, since it does not require us to 
identify the worst scenario. Using our methodology, Aggarwal [Agg94] discovered a bug 
in an algorithm designed by himself and Kutten [AK93] which was due to an argument of 
the kind cited above. Similarly, we discovered a bug in the timing analysis of the mutual 
exclusion algorithm of Pnueli and Zuck [PZ86]. This bug arose for the same reason. 


The reader familiar with existing work, and in particular familiar with model checking, may 
be a bit puzzled at this point. There is a considerable amount of work on model checking 
of randomized distributed systems, and yet we are introducing new techniques. Furthermore, 
although there is some ongoing work on automating part of the proof methods developed in this 
thesis [PS95], we do not address any decidability issue here. Our favorite analogy to justify our 
approach is that we view model checking as the program “Mathematica”, a popular program 
for symbolic manipulation of analytic expressions. If we are given a simple analytical problem, 
we can use Mathematica to get the solution from a computer. On the other hand, if we have 
a complex analytical problem, say a complex function that we have defined, and we want to 
verify that it respects some specific constraints, or maybe we want to find the constraints, then 
things are very different, since the problem in general is undecidable, i.e., not solvable by a 
computer. We can plot part of the given function using Mathematica and have a rough idea of 
whether it satisfies the desired constraints. If the plot shows that the function violates some 
of the constraints, then we have to change either the function or the constraints; if the plot 
shows that the function does not violate the constraints, then we can start to use all the tools 
of analysis to prove that the given function satisfies the constraints. In this way Mathematica 
saves us a lot of time. In using the analytical tools we need to use our creativity and our 
intuition about the problem so that we can solve its undecidable part. We view our research as 
building the analytical tools. 


Simulations 


The study of traces and simulations carried out in the thesis contributes more directly to theory 
than to practice. In particular, we do not give any examples of verification using simulations. 
However, due to the success that simulation relations have had for the verification of ordinary 
labeled transition systems, it is likely that the same methods will also work for randomized 
systems. 

A considerable amount of research has been carried out in extending trace semantics and 
simulation relations to the probabilistic case, especially within process algebras [Hoa85, Mil89, 
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BW90]; however, most of the existing literature does not address pure nondeterminism, and 
thus it has limited practical applicability. We believe it is important to have a model that is 
both useful for realistic problems and accounts for the existing theoretical work. In particu- 
lar, based on some of the interpretations that are given to nondeterminism within ordinary 
automata, we realize that, also in the probabilistic case, pure nondeterminism can be used to 
express much more than just the relative speeds of processes running concurrently. Specifically, 
nondeterminism can be used to model the following phenomena. 


1. Scheduling freedom. This is the classical use of nondeterminism, where several processes 
run in parallel and there is freedom in the choice of which process performs the next 
transition. 


2. External environment. Some of the labels can represent communication events due to the 
action of some external user, or more generally, to the action of an external environment. 
In this case nondeterminism models the arbitrary behavior of the external environment, 
which is chosen by an adversary. 


3. Implementation Freedom. A probabilistic automaton is viewed as a specification, and 
nondeterminism represents implementation freedom. That is, if from some state there 
are two transitions that can be chosen nondeterministically, then an implementation can 
have just one of the two transitions. In this case an adversary chooses the implementation 
that is used. 


It is important to recognize that, in the labeled transition system model, the three uses of 
nondeterminism described above can coexist within the same automaton. It is the specific 
interpretation that is given to the labels that determines what is expressed by nondeterminism 
at each point. 


1.2. Organization of the Thesis 


The thesis is divided in two main parts: the first part deals with the untimed model and the 
second part deals with the timed model. The second part relies heavily on the first part and 
adds a collection of results that are specific to the analysis of real-time properties. We describe 
the technical contributions of the thesis chapter by chapter. 


An Overview of Related Work. Chapter 2 gives an extensive overview of existing work 
on modeling and verification of randomized distributed systems. 


Preliminaries. Chapter 3 gives the basics of probability theory that are necessary to under- 
stand the thesis and gives an overview of the labeled transition systems model. All the topics 
covered are standard, but some of the notation is specific to this thesis. 


Probabilistic Automata. Chapter 4 presents the basic probabilistic model. A probabilistic 
automaton is a state machine whose transitions lead to a probability distribution over the labels 
that can occur and the new state that is reached. Thus, a transition describes the probabilistic 
behavior of a probabilistic automaton, while the choice of which transition to perform describes 
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the nondeterministic behavior of a probabilistic automaton. A computation of a probabilistic 
automaton, called a probabilistic execution, is the result of resolving the nondeterminism in a 
probabilistic automaton, i.e., the result of choosing a transition, possibly using randomization, 
from every point. A probabilistic execution is described essentially by an infinite tree with 
probabilities associated with its edges. On such a tree it is possible to define a probability 
space, which is the object through which the probabilistic properties of the computation can 
be studied. We extend the notions of finiteness, prefix and suffix of ordinary executions to 
the probabilistic framework and we extend the parallel composition operator. Finally, we show 
how to project a probabilistic execution of a compound probabilistic automaton onto one of 
its components and we show that the result is a probabilistic execution of the component. 
Essentially, we show that the properties of ordinary automata are preserved in the probabilistic 
framework. The probabilistic model is an extension of ordinary automata since an ordinary 
automaton can be viewed as a probabilistic automaton where each transition leads just to one 
action and one state. 


Direct Verification: Stating a Property. Chapter 5 shows how to formalize commonly 
used statements about randomized algorithms and shows how such formal statements can be 
manipulated. We start by formalizing the idea of an adversary, i.e., the entity that resolves 
the nondeterminism of a system in a malicious way. An adversary is a function that, given 
the past history of a system, chooses the next transition to be scheduled, possibly using ran- 
domization. The result of the interaction between an adversary and a probabilistic automaton 
is a probabilistic execution, on which it is possible to study probabilistic properties. Thus, 
given a collection of adversaries and a specific property, it is possible to establish a bound on 
the probability that the given property is satisfied under any of the given adversaries. We call 
such bound statements probabilistic statements. We show how probabilistic statements can be 
combined together to yield more complex statements, thus allowing for some form of compo- 
sitional verification. We introduce a special kind of probabilistic statement, called a progress 
statement, which is a probabilistic extension of the leads-to operator of UNITY [CM88]. Infor- 
mally, a progress statement says that if a system is started from some state in a set of states 
U, then, no matter what adversary is used, a state in some other set of states U’ is reached 
with some minimum probability p. Progress statements can be combined together under some 
general conditions on the class of adversaries that can be used. 

Finally, we investigate the relationship between deterministic adversaries (i.e., adversaries 
that cannot use randomness in their choices) and general adversaries. We show that for a large 
class of collections of adversaries and for a large class of properties it is sufficient to analyze 
only deterministic adversaries in order to derive statements that concern general adversaries. 
This result is useful in simplifying the analysis of a randomized algorithm. 


Direct Verification: Proving a Property. Chapter 6 shows how to prove the validity 
of a probabilistic statement from scratch. We introduce a collection of coin lemmas, which 
capture a common informal argument on probabilistic algorithms. Specifically, for many proofs 
in the literature the intuition behind the correctness of an algorithm is based on the following 
fact: if some specific random draws give some specific results, then the algorithm guarantees 
success. Then, the problem is reduced to showing that, no matter what the adversary does, 
the specific random draws give the specific results with some minimum probability. The coin 
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lemmas can be used to show that the specific random draws satisfy the minimum probability 
requirement; then, the problem is reduced to verifying properties of a system that does not 
contain probability at all. Factoring out the probability from a problem helps considerably in 
removing errors due to unexpected dependencies. 

We illustrate the method by verifying the correctness of the randomized dining philosophers 
algorithm of Lehmann and Rabin [LR81] and the algorithm for randomized agreement with 
stopping faults of Ben-Or [BO83]. In both cases the correctness proof is carried out by proving 
a collection of progress statements using some coin lemmas. 

Finally, we suggest another technique, called the partition technique, that departs consid- 
erably from the coin lemmas and that appears to be useful in some cases. We illustrate the 
partition technique on a toy resource allocation protocol, which is one of the guiding examples 
throughout Chapters 5 and 6. 


Hierarchical Verification: Trace Distributions. Chapter 7 extends the trace-based se- 
mantics of ordinary automata [Hoa85] to the probabilistic framework. A trace is a ordered 
sequence of labels that occur in an execution; a trace distribution is the probability distribu- 
tion on traces induced by a probabilistic execution. We extend the trace preorder of ordinary 
automata (inclusion of traces) to the probabilistic framework by defining the trace distribution 
preorder. However, the trace distribution preorder is not preserved by the parallel composition 
operator, i.e., it is not a precongruence. Thus, we define the trace distribution precongruence 
as the coarsest precongruence that is contained in the trace distribution preorder. Finally, we 
show that there is an elementary probabilistic automaton called the principal context that dis- 
tinguishes all the probabilistic automata that are not in the trace distribution precongruence 
relation. This leads us to an alternative characterization of the trace distribution precongruence 
as inclusion of principal trace distributions. 


Hierarchical Verification: Simulations. Chapter 8 extends the verification method based 
on simulation relations to the probabilistic framework. Informally, a simulation relation from 
one automaton to another automaton is a relation between the states of the two automata that 
allows us to embed the transition relation of one automaton in the other automaton. In the 
probabilistic framework a simulation relation is still a relation between states; however, since 
a transition leads to a probability distribution over states, in order to say that a simulation 
relation embeds the transition relation of a probabilistic automaton into another probabilistic 
automaton we need to extend a relation defined over states to a relation defined over probabil- 
ity distributions over states. We generalize the strong and weak bisimulation and simulation 
relations of Milner, Jonsson, Lynch and Vaandrager [Mil89, Jon91, LV91] to the probabilistic 
framework. Then, we introduce a coarser simulation relation, called a probabilistic forward 
simulation, where a state is related to a probability distribution over states rather than to a 
single state. We prove an execution correspondence theorem which, given a simulation relation 
from one probabilistic automaton to another probabilistic automaton, establishes a strong cor- 
respondence between each probabilistic execution of the first probabilistic automaton and one 
of the probabilistic executions of the second automaton. Based on the execution correspon- 
dence theorem, we show that each of the relations presented in the chapter is sound for the 
trace distribution precongruence. Thus, simulation relations can be used as a sound technique 
to prove principal trace distribution inclusion. 
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Probabilistic Timed Automata. Chapter 9 starts the second part of the thesis. We extend 
probabilistic automata with time following the approach of Lynch and Vaandrager [LV95], where 
passage of time is modeled by means of transitions labeled with positive real numbers. In order 
to use most of the untimed theory, we force time-passage transition not to be probabilistic. 
We extend probabilistic executions to the timed framework, leading to probabilistic timed 
executions, and we show the relationship between probabilistic executions and probabilistic 
timed executions. The main idea is that in several circumstances it is sufficient to analyze the 
probabilistic executions of a system in order to study its real-time behavior. 


Direct Verification: Time Complexity. Chapter 10 introduces new techniques for the 
verification of real-time properties of a randomized algorithm. The techniques of Chapter 5 
still apply; however, due to the presence of time, it is possible to study the time complexity 
of an algorithm. We augment the progress statements of Chapter 5 with an upper bound ¢ to 
state the following: if a system is started from some state in a set of states U, then, no matter 
what adversary is used, a state of some other set of states U’ is reached within time ¢ with 
some minimum probability p. Based on these timed progress statements, we show how to derive 
upper bounds on the expected time to reach some set of states. We illustrate the technique 
by showing that the randomized dining philosophers algorithm of Lehmann and Rabin [LR81] 
guarantees progress within expected constant time. 

By extending the technique for the analysis of expected time, we show how to derive bounds 
on more abstract notions of complexity. In particular, we consider the algorithm for randomized 
agreement of Ben-Or as an example. The algorithm of Ben-Or runs in stages. From the way 
the algorithm is structured, it is not possible to give meaningful bounds on the time it takes 
to make progress from any reachable state. However, using abstract complexities, it is easy 
to prove an upper bound on the expected number of stages that are necessary before reaching 
agreement. Once an upper bound on the expected number of stages is derived, it is easy to 
derive an upper bound on the expected time to reach agreement. 


Hierarchical Verification: Timed Trace Distributions and Timed Simulations. Chap- 
ters 11 and 12 extend the trace distribution precongruence and the simulation relations of the 

untimed framework to the timed framework. A trace is replaced by a timed trace, where a 

timed trace is a sequence of labels paired with their time of occurrence plus a limit time. The 

timed trace distribution precongruence is characterized by a timed principal context, which is 

the principal context augmented with arbitrary time-passage transitions. All the timed simu- 

lation relations are shown to be sound for the timed trace distribution precongruence. All the 

results are proved by reducing the problem to the untimed framework. 


Conclusion. Chapter 13 gives some concluding remarks and several suggestions for further 
work. Although this thesis builds a model for randomized computation and shows that it is 
sufficiently powerful for the analysis of randomized distributed real-time algorithms, it just 
discovers the tip of the iceberg. We propose a methodology for the analysis of randomization, 
and we give several examples of the application of such methodology; however, there are several 
other ways to apply our methodology. It is very likely that new probabilistic statements, new 
results to combine probabilistic statements, and new coin lemmas can be developed based on the 
study of other algorithms; similarly, the fundamental idea behind the trace semantics that we 
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present can be used also for other kinds of observational semantics like failures [Hoa85, DH84]. 
We give hints on how it is possible to handle liveness within our model and state what we know 
already. Furthermore, we give ideas of what is possible within restricted models where some 
form of I/O distinction like in the work of Lynch and Tuttle [LT87] or some timing restriction 
like in the work of Merritt, Modugno and Tuttle [MMT91] is imposed. Finally, we address the 
issue of relaxing some of the restrictions that we impose on the timed model. 


1.3 Reading the Thesis 


The two parts of the thesis, the untimed and the timed part, proceed in parallel: each chapter of 
the untimed part is a prerequisite for the corresponding chapter in the timed part. Each part is 
subdivided further into two parts: the direct verification and the hierarchical verification. The 
two parts can be read almost independently, although some knowledge of the direct verification 
method can be of help in reading the hierarchical method. The direct method is focused mainly 
on verification of algorithms, while the hierarchical method is focused mainly on the theoretical 
aspects of the problem. Further research should show how the hierarchical method can be of 
significant help for the analysis of randomized algorithms. 

Each chapter starts with an introductory section that gives the main motivations and an 
overview of the content of the chapter. Usually, the more technical discussion is concentrated 
at the end. The same structure is used for each section: the main result and short proofs are 
at the beginning of each section, while the long proofs and the more technical details are given 
at the end. A reader can skip the proofs and the most technical details on a first reading in 
order to have a better global picture. It is also possible to read just Chapter 3 and the first 
section (including subsections) of Chapters 4 to 12, and have a global view of the results of 
the thesis. In a second reading, the interested reader can concentrate on the proofs and on the 
technical definitions that are necessary for the proofs. The reader should keep in mind that 
several proofs in the thesis are based on similar techniques. Such techniques are explained in 
full detail only the first time they are used. 

A reader interested only in the techniques for the direct verification of algorithms and not 
interested in the arguments that show the foundations of the model can avoid reading the proofs. 
Moreover, such a reader can just glance over Section 4.2.6, and skip Sections 4.2.7, 4.3, and 4.4. 
In the timed framework the reader interested just in the techniques for the direct verification 
of algorithms can skip all the comparison between the different types of probabilistic timed 
executions and concentrate more on the intuition behind the definition of a probabilistic timed 
execution. 
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Chapter 2 


An Overview of Related Work 


In this chapter we give an extensive overview of existing work on modeling and verification of 
randomized distributed systems. We defer the comparison of our work with the existing work 
to the end of each chapter. Some of the descriptions include technical terminology which may 
be difficult to understand for a reader not familiar with concurrency theory. Such a reader 
should focus mainly on the high level ideas and not worry about the technical details. The rest 
of the thesis presents our research without assuming any knowledge of concurrency theory. We 
advise the reader not familiar with concurrency theory to read this chapter again after reading 
the thesis. 

There have been two main research directions in the field of randomized distributed real-time 
systems: one focused mainly on modeling issues using process algebras [Hoa85, Mil89, BW90] 
and labeled transition systems [Kel76, Plo81] as the basic mathematical objects; the other 
focused mainly on verification using Markov chains as the basic model and temporal logic 
arguments [Pnu82] and model checking [EC82, CES83] as the basic verification technique. Most 
of the results of the first of the research directions fail to model pure nondeterminism, while 
the results of the second of the research directions model pure nondeterminism successfully, but 
not in its full generality. As expressed at the end of Section 1.1.2, pure nondeterminism arises 
only in the choice of what process is performing the next instruction at each moment. Below 
we summarize the results achieved in both of the research directions. Furthermore, at the end 
of each chapter we add a section where we explain how the results described in this section are 
related to our research. 


2.1 Reactive, Generative and Stratified Models 


We present some of the existing work on modeling which is based on a classification due to van 
Glabbeek, Smolka, Steffen and Tofts [GSST90]. They define three types of processes: reactive, 
generative, and stratified. 


e Reactive model: Reactive processes consist of states and labeled transitions associated 
with probabilities. The restriction imposed on a reactive process is that for each state the 
sum of the probabilities of the transitions with the same label is 1. 


e Generative model: Generative processes consist of states and labeled transitions associated 
with probabilities. The restriction imposed on a generative process is that for each state 
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Figure 2-1: Reactive, generative and stratified processes, from left to right. 


either there are no outgoing transitions, or the sum of the probabilities of all the outgoing 
transitions is 1. 


e Stratified model: Stratified processes consist of states, unlabeled transitions associated 
with probabilities, and labeled transitions. The restriction imposed on a stratified process 
is that for each state either there is exactly one outgoing labeled transition, or all the 
outgoing transitions are unlabeled and the sum of their probabilities is 1. 


Figure 2-1 gives an example of a reactive, a generative, and a stratified process. Informally, 
reactive processes specify for each label (also called action) the probability of reaching other 
states; generative processes also give additional information concerning the relative probabili- 
ties of the different actions; stratified processes add some probabilistic structure to generative 
processes. Observe that among the three models above only the reactive model has a struc- 
ture that can be used to express some form of pure nondeterminism (what action to perform), 
although in van Glabbeek et al. [GSST90] this issue is not considered. 


2.1.1 Reactive Model 


Rabin [Rab63] studies the theory of probabilistic automata, which are an instance of the reactive 
model. He defines a notion of a language accepted by a probabilistic automaton relative to a 
cut point A and shows that there are finite state probabilistic automata that define non-regular 
languages. 

Larsen and Skou [LS89, LS91] define a bisimulation type semantics, called probabilistic 
bisimulation, and a logic, called probabilistic model logic (PML), for reactive processes, and 
they introduce a notion of testing based on sequential tests and a copying facility. They show 
that two processes that satisfy the minimal probability assumption are probabilistically bisim- 
ilar if and only if they satisfy exactly the same PML formulas, and that two processes that 
satisfy the minimal probability assumption and that are not probabilistically bisimilar can be 
distinguished through testing with a probability arbitrarily close to 1. The minimum proba- 
bility assumption states that for every state the probability of each transition is either 0 or is 
above some minimal value. This condition corresponds to the image-finiteness condition for 
non-probabilistic processes. Bloom and Meyer [BM89] relate the notions of probabilistic and 
non-probabilistic bisimilarity by showing that two non-probabilistic finitely branching processes 
P and @ are bisimilar if and only if there exists an assignment of probabilities to the transi- 
tions of P and Q such that the corresponding reactive processes P’ and Q’ are probabilistically 
bisimilar. 

Larsen and Skou [L592] introduce a synchronous calculus for reactive processes where the 
probabilistic behavior is obtained through a binary choice operator parameterized by a prob- 
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ability p. They define a bisimulation relation on the new calculus, and they introduce a new 
extended probabilistic logic (EPL) which extends PML in order to support decomposition with 
respect to parallel composition. Both the probabilistic bisimulation and the extended proba- 
bilistic logic are axiomatized. 


2.1.2 Generative and Stratified Models 


Giacalone, Jou and Smolka [GJS90] define a process algebra for generative processes, called 
PCCS, which can be seen as a probabilistic extension of Milner’s SCCS [Mil93]. In PCCS two 
processes synchronize at every transition regardless of the action that they perform. That is, if 
one process performs a transition labeled with action a with probability p, and another process 
performs a transition labeled with 6 with probability p,, then the two processes together can 
perform a transition labeled with ab with probability p,p,. The authors provide an equational 
theory for PCCS based on the probabilistic bisimulation of Larsen and Skou [LS89], and provide 
an axiomatization for probabilistic bisimulation (the axiomatization is shown to be sound and 
complete in [JS90]). Furthermore, the authors define a notion of ¢bisimulation, where two 
processes can simulate each other’s transition with a probability difference at most ¢«. Based on 
e-bisimulation, the authors define a metric on generative processes. 

Jou and Smolka [JS90] define trace and failure equivalence for generative processes. They 
show that, unlike for nondeterministic transition systems, maximality of traces and failures does 
not increase the distinguishing power of trace and failure equivalence, where by maximality of 
a trace we mean the probability to produce a specific trace and then terminate. More precisely, 
knowing the probability of each finite trace of a generative process gives enough information to 
determine the probability that a finite trace occurs leading to termination; similarly, knowing 
the probability of every failure of a generative process gives enough information to determine 
the probability of each maximal failure. Jou and Smolka show also that the trace and failure 
equivalences are not congruences. Our probabilistic executions are essentially generative pro- 
ceses, and our trace distributions are essentially the trace semantics of Jou and Smolka. In our 
case the properties shown by Jou and Smolka follow directly from measure theory. 

Van Glabbeek et al. [GSST90] state that the generative model is more general than the 
reactive model in the sense that generative processes, in addition to the relative probabilities 
of transitions with the same label, contain information about the relative probabilities of tran- 
sitions with different labels. They show also that the stratified model is a generalization of the 
generative model in the sense that a probabilistic choice in the generative model is refined by 
a structure of probabilistic choices in the stratified model. Formally, the authors give three 
operational semantics to PCCS, one reactive, one generative, and one stratified, and show how 
to project a stratified process into a generative process and how to project a generative process 
into a reactive process, so that the operational semantics of PCCS commute with the projec- 
tions. The reactive and generative processes of Figure 2-1 are the result of the projection of 
the generative and stratified processes, respectively, of Figure 2-1. Finally, the authors define 
probabilistic bisimulation for the generative and for the stratified models and show that bisim- 
ulation is a congruence in all the models and that bisimulation is preserved under projection 
from one model to the other. The results of van Glabbeek et al. [GSST90], however, are based 
on the fact that parallel composition is synchronous. 

Tofts [Tof90] introduces a weighted synchronous calculus whose operational semantics resem- 


25 


bles the stratified model. The main difference is that the weights associated with the transitions 
are not probabilities, but rather frequencies, and thus their sums are not required to be 1. Tofts 
defines two bisimulation relations that are shown to be congruences. The first relation is sensi- 
tive to the actual frequencies of the transitions leaving from a state, while the second relation 
is sensitive only to the relative frequencies of the transitions leaving from a state. In particular, 
the second relation coincides with the stratified bisimulation of van Glabbeek et al. [GSST90] 
after normalizing to 1 the frequencies of the transitions that leave from every state. The ad- 
vantage of Tofts’ calculus is that it is not necessary to restrict the syntax of the expressions so 
that the weights of the choices at any point sum to | (such a restriction is imposed in PCCS). 
Moreover, it is possible to define a special weight w that expresses infinite frequency and can 
be used to express priorities. A similar idea to express priorities is used by Smolka and Steffen 
in [SS90], where the stratified semantics of PCCS is extended with 0-probability transitions. 

Baeten, Bergstra and Smolka [BBS92] define an algebra, prACP;, which is an extension 
of ACP [BW90] with generative probabilities. The authors show that prACP7 and a weaker 
version of ACP (ACP7) are correlated in the sense that ACP; is the homomorphic image 
of prACP; in which the probabilities are forgotten. The authors also provide a sound and 
complete axiomatization of probabilistic bisimulation. 

Wu, Smolka and Stark [WSS94] augment the I/O automaton model of Lynch and Tuttle 
[LT87] with probability and they study a compositional behavioral semantics which is also 
shown to be fully abstract with respect to probabilistic testing. A test is a probabilistic I/O 
automaton with a success action w. The model is reactive for the input actions and generative 
for the output actions. This allows the authors to define a meaningful parallel composition 
operator, where two probabilistic I/O automata synchronize on their common actions and 
evolve independently on the others. In order to deal with the nondeterminism that arises from 
parallel composition, the authors attach a delay parameter to each state of a probabilistic I/O 
automaton, which can be seen as the parameter of an exponential probability distribution on 
the time of occurrence of the next local (i.e., output or internal) action. Whenever there is a 
conflict for the occurrence of two local actions of different probabilistic I/O automata, the delay 
parameters associated with the states are used to determine the probability with which each 
action occurs. The behavior of a probabilistic I/O automaton A is a function €4 that associates 
a functional E} with each finite trace 9 . If the length of § is n, then E} takes a function f 
that given n+1 delay parameters computes an actual delay, and returns the expected value of 
f applied to the delay parameters of the computations of A that lead to @. 


2.2 Models based on Testing 


Research on modeling has also focused on extending the testing preorders of De Nicola and 
Hennessy [DH84] to probabilistic processes. To define a testing preorder it is necessary to 
define a notion of a test and of how a test interacts with a process. The interaction between 
a test and a process may lead to success or failure. Then, based on the success or failure of 
the interactions between a process and a test, a preorder relation between processes is defined. 
Informally, a test checks whether a process has some specific features: if the interaction between 
a test and a process is successful, then the process has the desired feature. 

Ivan Christoff [Chr90b, Chr90a] analyzes generative processes by means of testing. A test 
is a nondeterministic finite-state process, and the interaction between a process and a test is 
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obtained by performing only those actions that both the processes offer and by keeping the 
relative probability of each transition unchanged. Four testing preorders are defined, each one 
based on the probability of the traces of the interaction between a process and a test. Christoff 
also provides a fully abstract denotational semantics for each one of the testing preorders: each 
process is denoted by a mapping that given an offering and a trace returns a probability. An 
offering is a finite sequence of non-empty sets of actions, and, informally, describes the actions 
that the environment offers to a process during the interaction between the process and a test. 

Linda Christoff [Chr93] builds on the work of Ivan Christoff and defines three linear se- 
mantics for generative processes: the trace semantics, the broom semantics, and the barbed 
semantics. The relations are defined in a style similar to the denotational models of Ivan 
Christoff, and, in particular, the trace and barbed semantics coincide with two of the semantics 
of [Chr90b]. Linda Christoff also defines three linear-time temporal logics that characterize her 
three semantics and provides efficient model checking algorithms for the recursion-free version 
of the logics. 

Testing preorders that are more in the style of De Nicola and Hennessy [DH84] are presented 
by Yi and Larsen in [YL92], where they define a process algebra with all the operators of CCS 
plus a binary probabilistic choice operator parameterized by a probability p. Thus, the calculus 
of Yi and Larsen allows for nondeterminism. A test is a process of their calculus with an 
additional label w. Depending on how the nondeterminism is resolved, w occurs with different 
probabilities in the interaction between a process and a test. Then, Yi and Larsen define a may 
preorder, which is based on the highest probability of occurrence of w, and a must preorder, 
which is based on the lowest probability of occurrence of w. The two preorders are shown to 
coincide with the testing preorders of De Nicola and Hennessy [DH84] when no probability is 
present. In more recent work Jonsson, Ho-Stuart and Yi [JHY94] give a characterization of 
the may preorder based on tests that are not probabilistic, while Jonsson and Yi [JY95] give a 
characterization of the may and must preorders based on general tests. 

Cleaveland, Smolka and Zwarico [CSZ92] introduce a testing preorder on reactive processes. 
A test is a reactive process with a collection of successful states and a non-observable action. 
The interaction between a test and a process allows an observable action to occur only if 
the two processes allow it to occur, and allows the non-observable action to occur if the test 
allows it to occur. The result is a generative process, where each of the actions that occur is 
chosen according to a uniform distribution (thus the formalism works only for finitely many 
actions). Two processes are compared based on the probability of reaching a successful state in 
the interaction between a process and a test. The authors show that their testing preorder is 
closely connected to the testing preorders of De Nicola and Hennessy [DH84] in the sense that 
if a process passes a test with some non-zero probability, then the non-probabilistic version 
of the process (the result of removing the probabilities from the transition relation of the 
process) may pass the non-probabilistic version of the test, and if a process passes a test with 
probability 1, then the non-probabilistic version of the process must pass the non-probabilistic 
version of the test. An alternative characterization of the testing preorder of Cleaveland et al. 
[CSZ92] is provided by Yuen, Cleaveland, Dayar and Smolka [YCDS94]. A process is represented 
as a mapping from probabilistic traces to [0,1], where a probabilistic trace is an alternating 
sequence of actions and probability distributions over actions. Yuen et al. use the alternative 
characterization to show that the testing preorder of Cleaveland et al. [CSZ92] is an equivalence 
relation. 
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2.3. Models with Nondeterminism and Denotational Models 


2.3.1 Transitions with Sets of Probabilities 


Jonsson and Larsen [JL91] introduce a new kind of probabilistic transition system where the 
transitions are labeled by sets of allowed probabilities. The idea is to model specifications where 
the probabilities associated with the transitions are not completely specified. They extend the 
bisimulation of Larsen and Skou [LS89] to the new framework and they propose two criteria for 
refinement between specifications. One criterion is analogous to the definition of simulations 
between non-probabilistic processes; the other criterion is weaker and regards a specification 
as a set of probabilistic processes. Refinement is then defined as inclusion of probabilistic 
processes. Finally, Jonsson and Larsen present a complete method for verifying containment 
between specifications. 


2.3.2 Alternating Models 


Hansson and Jonsson [HJ89, HJ90] develop a probabilistic process algebra based on an alternat- 
ing model. The model of Hansson and Jonsson, which is derived from the Concurrent Markov 
Chains of Vardi [Var85], is a model in which there are two kinds of states: probabilistic states, 
whose outgoing transitions are unlabeled and lead to nondeterministic states, and nondetermin- 
istic states, whose outgoing transitions are labeled and lead to probabilistic states. Only the 
transitions leaving from probabilistic states are probabilistic, and for each probabilistic state 
the probabilities of the outgoing transitions add to 1. The authors define a strong bisimulation 
semantics in the style of Larsen and Skou [LS89] for which they provide a sound and complete 
axiomatization. The model of Hansson and Jonsson [HJ90] differs substantially from the models 
of van Glabbeek et al. [GSST90] in that there is a clear distinction between pure nondeterminism 
and probability. The model could be viewed as an instance of the reactive model; however, the 
parallel composition operation defined by Hansson and Jonsson [HJ90] is asynchronous, while 
the classification of van Glabbeek et al. [GSST90] works only for synchronous composition. A 
complete presentation of the work of Hansson and Jonsson [HJ89, HJ90] appears in Hansson’s 
PhD thesis [Han91], later published as a book [Han94]. Our simple probabilistic automata are 
very similar in style to the objects of Hansson’s book. 


2.3.3. Denotational Semantics 


Seidel [Sei92] extends CSP [Hoa85] with probability. The extension is carried out in two steps. 
In the first step a process is a probability distribution over traces; in the second step, in order 
to account for the nondeterministic behavior of the environment, a process is a conditional 
probability measure, i.e., an object that given a trace, which is meant to be produced by the 
external environment, returns a probability distribution over traces. 

Jones and Plotkin [JP89] use a category theoretic approach to define a probabilistic pow- 
erdomain, and they use it to give a semantics to a language with probabilistic concurrency. 
It is not known yet how the semantics of Jones and Plotkin compares to existing operational 
semantics. 


28 


2.4 Models with Real Time 


There are basically two models that address real time issues. One model is the model of Hansson 
and Jonsson [Han94], where special y actions can appear in the transitions. The occurrence of 
an action y means that time has elapsed, and the amount of time that elapses in a computation 
is given by the number of occurrences of action y. Thus, the time domain of Hansson and 
Jonsson’s model is discrete. 

The other model is based on stochastic process algebras and is used in the field of performance 
analysis. In particular, actions are associated with durations, and the durations are expressed 
by random variables. In order to simplify the analysis, the random variables are assumed to have 
an exponential probability distribution, which is memoryless. Research in this area includes 
work from Gétz, Herzog and Rettelbach [GHR93], from Hillston [Hil94], and from Bernardo, 
Donatiello and Gorrieri [BDG94]. 


2.5 Verification: Qualitative and Quantitative Methods 


Most of the research on the verification of randomized distributed systems is concerned with 
properties that hold with probability 1. The advantage of such properties is that for finite 
state processes they do not depend on the actual probabilities of the transitions, but rather on 
whether those transitions have probability 0 or probability different from 0. Thus, the problem 
of checking whether a system satisfies a property with probability 1 is reduced to the problem 
of checking whether a non-randomized system satisfies some other property. This method is 
called qualitative, as opposed to the quantitative method, where probabilities different from 1 
also matter. 

The rationale behind the qualitative method is that a randomized process, rather than 
always guaranteeing success, usually guarantees success with probability 1, which is practically 
the same as guaranteeing success always. The quantitative method becomes relevant whenever 
a system has infinitely many states or the complexity of an algorithm needs to be studied. 

Almost all the papers that we describe in this section are based on a model where n Markov 
chains evolve concurrently. Each Markov chain represents a process, and the pure nondeter- 
minism arises from the choice of what Markov chain performs the next transition (what process 
is scheduled next). The object that resolves the nondeterminism is called a scheduler or adver- 
sary, and the result of a scheduler on a collection of concurrent Markov chains is a new Markov 
chain that describes one of the possible evolutions of the global system. Usually a scheduler is 
required to be fair in the sense that each process should be scheduled infinitely many times. 


2.5.1 Qualitative Method: Proof Techniques 


Huart, Sharir and Pnueli [HSP83] consider n finite state asynchronous randomized processes 
that run in parallel, and provide two necessary and sufficient conditions to guarantee that a 
given set of goal states is reached with probability 1 under any fair scheduler. A scheduler is 
the entity that at any point chooses the next process that performs a transition. The result 
of the action of a scheduler on n processes is a Markov chain, on which it is possible to study 
probabilities. A scheduler is fair if and only if, for each path in the corresponding Markov 
chain, each process is scheduled infinitely many times. The authors show that in their model 
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each property described by reaching a collection of states has either probability 0 or probability 
1. Then, they describe a decision procedure for the almost sure reachability of a set of goal 
states. The procedure either constructs a decomposition of the state space into a sequence of 
components with the property that any fair execution of the program must move down the 
sequence with probability 1 until it reaches the goal states (goal states reached with probability 
1), or finds an ergodic set of states through which the program can loop forever with probability 
1 (goal states reached with probability 0). Finally the authors give some examples of problems 
where the use of randomization does not provide any extra power over pure nondeterminism. 
The proof principle of [HSP83] is generalized to the infinite state case by Hart and Sharir 
[HS85]. 

Lehmann and Shelah [L582] extend the temporal logic of linear time of Pnueli [Pnu82] to 
account for properties that hold with probability 1, and they provide three complete axioma- 
tizations of the logic: one axiomatization is for general models, one is for finite models, and 
one is for models with bounded transition probabilities (same as the minimum probability re- 
quirement of Larsen and Skou [L591]). A model of the logic is essentially a Markov chain, 
or alternatively an unlabeled generative process. The logic of Lehmann and Shelah [LS82] is 
obtained from the logic of Pnueli [Pnu82] by adding a new modal operator V whose meaning 
is that the argument formula is satisfied with probability 1. 

Pnueli [Pnu83] introduces the notion of extreme fairness and shows that a property that 
holds for all extreme fair executions holds with probability 1. Furthermore, Pnueli presents a 
sound proof rule based on extreme fairness and linear temporal logic. The model consists of n 
randomized processes in parallel. Each process is a state machine where each state enables a 
probabilistic transition, which lead to several modes. Resolving the nondeterminism leads to a 
Markov chain. However, only those Markov chains that originate from fair scheduling policies 
are considered. Then, an execution (a path in the Markov chain) is extremely fair relative 
to a property ¢ (¢ is a property that is satisfied by states) if and only if for each transition 
that occurs infinitely many times from states that satisfy @, each mode of the transition occurs 
infinitely many times. An execution is extremely fair if and only if it is extremely fair relative 
to any formula @ expressed in the logic used in [Pnu83]. The proof rule of Pnueli [Pnu83], 
along with some other new rules, is used by Pnueli and Zuck [PZ86] to verify two non-trivial 
randomized algorithms, including the Randomized Dining Philosophers algorithm of Lehmann 
and Rabin [LR81]. Zuck [Zuc86] introduces the notion of a-fairness and shows that a-fairness 
is complete for temporal logic properties that hold with probability 1. 

Rao [Rao90] extends UNITY [CM88] to account for randomized systems and properties 
that hold with probability 1. The main emphasis is on properties rather than states. A new 
notion of weak probabilistic precondition is introduced that, together with the extreme fairness 
of Pnueli, generalizes weakest preconditions. Finally, based on the work of Huart et al. [HSP83], 
Rao argues that his new logic is complete for finite state programs. 


2.5.2 Qualitative Method: Model Checking 


Vardi [Var85] presents a method for deciding whether a probabilistic concurrent finite state 
program satisfies a linear temporal logic specification, where satisfaction means that a formula 
is satisfied with probability 1 whenever the scheduler is fair. A program is given as a Concurrent 
Markov Chain, which is a transition system with nondeterministic and probabilistic states. A 
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subset fF of the nondeterministic states is called the set of fair states. A scheduler is a function 
that, based on the past history of a program, chooses the next transition to perform from 
a nondeterministic state. The result of the action of a scheduler on a program is a Markov 
chain on which it is possible to study the probability that some linear temporal logic formula 
is satisfied. A path in the Markov chain is fair if for each fair state that occurs infinitely many 
times each one of the possible nondeterministic choices from that state occurs infinitely many 
times; a scheduler is fair if the fair paths have probability 1 in the corresponding Markov chain. 
The model checking algorithm of Vardi works in time polynomial in the size of the program and 
doubly exponential in the size of the specification. By considering a slightly restricted logic, 
Vardi and Wolper [VW86] reduce the complexity of the model checking algorithm to only one 
exponent in the size of the formula. 

Courcoubetis and Yannakakis [CY88, CY90] investigate the complexity of model checking 
linear time propositional temporal logic of sequential and concurrent probabilistic processes. A 
sequential process is a Markov chain and a concurrent process is a Concurrent Markov Chain. 
They give a model checking algorithm that runs in time linear in the size of the program and 
exponential in the size of the formula, and they show that the problem is in PSPACE. Moreover, 
they give an algorithm for computing the exact probability with which a sequential program 
satisfies a formula. 

Alur, Courcoubetis and Dill [ACD91a, ACD91b] develop a model checking algorithm for 
probabilistic real-time systems. Processes are modeled as a generalized semi-Markov process, 
which are studied in [Whi80, She87]. Essentially a process is a finite state transition system 
with timing constraints expressed by probability distributions on the delays. They impose the 
restriction that every distribution is either discrete, or exponential, or has a density function 
which is different from 0 only on a finite collection of intervals (in [ACD91a] only this last case 
is studied). The temporal logic, called TCTL, is an extension of the branching-time temporal 
logic of Emerson and Clarke [EC82] where time delays are added to the modal operators. TCTL 
can detect only whether a formula is satisfied with probability 0, or with a positive probability, 
or with probability 1. The model checking algorithm transforms a process into a finite state 
process without probabilities and real-time, thus allowing the use of other existing algorithms. 


The problem of model-checking for TCTL is PSPACE-hard. 


2.5.3. Quantitative Method: Model Checking 


Hansson [Han91, Han94] defines a model checking algorithm for his Labeled Concurrent Markov 
Chain model and his branching-time temporal logic TPCTL. Time is discrete in Hansson’s 
model, but the logic improves on previous work because probabilities can be quantified (i.e., 
probabilities can be between 0 and 1). The previous model checking algorithms relied heavily 
on the fact that probabilities were not quantified. The algorithm is based on the algorithm 
for model checking of Clarke, Emerson and Sistla [CES83], and on previous work of Hansson 
and Jonsson [HJ89] where a model checking algorithm for PCTL (TPCTL without time) is 
presented. In order to deal with quantified probabilities, the algorithm reduces the computation 
of the probability of an event to a collection of finitely many linear recursive equations. The 
algorithm has an exponential complexity; however, Hansson shows that for a large class of 
interesting problems the algorithm is polynomial. 
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Chapter 3 


Preliminaries 


3.1 Probability Theory 


The rigorous study of randomized algorithms requires the use of several probability measures. 
This section introduces the basic concepts of measure theory that are necessary. Most of the 
results are taken directly from Halmos [Hal50] and Rudin [Rud66], and the proofs can be found 
in the same books or in any other good book on measure theory or probability theory. 


3.1.1 Measurable Spaces 


Consider a set 2. A field on 0, denoted by F’, is a family of subsets of Q that contains 0, and 
that is closed under complementation and finite union. A o-field on 0, denoted by 7, is a field 
on 2 that is closed under countable union. The elements of a o-field are called measurable sets. 
The pair (Q, F) is called a measurable space. 

A field generated by a family of sets C, denoted by F(C), is the smallest field that contains 
C. The o-field generated by a family of sets C, denoted by o(C), is the smallest o-field that 
contains C. The family C is called a generator for o(C). A trivial property of a generator C is 
o(C) = o(F(C)). 


The field generated by a family of sets can be obtained following a simple procedure. 
Proposition 3.1.1 Let C be a family of subsets of Q. 
1. Let F\(C) be the family containing 0, Q, and all C CQ such that CEC or (Q-C)EC. 
2. Let F2(C) be the family containing all finite intersections of elements of F\(C). 
3. Let F3(C) be the family containing all finite unions of disjoint elements of F2(C). 
Then F(C) = £3(C). = 


3.1.2. Probability Measures and Probability Spaces 


Let C be a family of subsets of 2. A measure yw on C is a function that assigns a non-negative 
real value (possibly oo) to each element of C, such that 


1. if @ is an element of C, then p(Q) = 0. 
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2. if (Ci)ien forms a sequence of pairwise disjoint elements of C, and U;C; is an element of 
C, then p(U;Ci) = 0; (Ci). 
The last property is called o-additivity. If (Q,F) is a measurable space, then a measure on F 
as called a measure on (Q, F). 

A measure on a family of sets C is finite if the measure of each element of C is finite. 

A measure space is a triple (Q, F, 4), where (Q, F) is a measurable space, and y is a measure 
on (Q, F). A measure space (Q, F, js) is complete iff for each element C' of F¥ such that u(C) = 0, 
each subset of Cis measurable and has measure 0, i.e., for each C’ C C, C’ € F and pu(C’) = 0. 
A measure space is discrete if F is the power set of Q and the measure of each measurable set 
is the sum of the measures of its points. Discrete spaces will play a fundamental role in our 
theory. 

A probability space is a triple (0,F,P), where (Q,F) is a measurable space, and P is a 
measure on (Q2,F) such that P(Q) = 1. The measure P is also referred to as a probability 
measure or a probability distribution. The set Q is called the sample space, and the elements 
of F are called events. We denote a generic event by F£, possibly decorated with primes and 
indices. A standard convention with probability measures and event is that the measure of an 
event is denoted by P[F] rather than by P(E). 


3.1.3. Extensions of a Measure 


The following two theorems shows methods to extend a measure defined on a collection of sets. 
The first theorem says that it is possible to define a probability measure P on a measurable 
space (Q,F) by specifying P only on a generator of F; the second theorem states that every 
measure space can be extended to a complete measure space. 

Thus, from the first theorem we derive that in order to check the equality of two probability 
measures P; and P2 on (Q, F), it is enough to compare the two measures on a field that generates 


F. 


Theorem 3.1.2 (Extension theorem) A finite measure : on a field F has a unique exten- 
sion to the o-field generated by F'. That is, there exists a unique measure [i on o( f°’) such that 


for each element C of F, f(C) = pC). = 


Theorem 3.1.3 Let (Q,F,) be a measure space. Let F' be the set of subsets of Q of the form 
CUN such that C € F and N is a subset of a set of measure 0 in F. Then, F' is a o-field. 
Furthermore, the function yi’ defined by p'(C UN) = p(C) is a complete measure on F', We 
denote the measure space (Q, F’, 11’) by completion((Q, F, 1)). a 


3.1.4 Measurable Functions 


Let (Q,F) and (Q’,F’) be two measurable spaces. A function f : Q2 — ! is said to be a 
measurable function from (Q,F) to (Q',F") if for each set C of F’ the inverse image of C, 
denoted by f~!(C), is an element of F. The next proposition shows that the measurability of 
f can be checked just by analyzing a generator of F’. 


Proposition 3.1.4 Let (Q,F) and (, F') be two measurable spaces, and let C be a generator 
of F'. Let f be a function form Q to Q’. Then f is measurable iff for each element C of C, the 
inverse image f—'(C) is an element of F. a 
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Another property that we need is the closure of measurable functions under composition. 


Proposition 3.1.5 Let f be a measurable function from (Q1, F1) to (Q2, F2), and let g be a 
measurable function from (Q2, Fz) to (Q3, F3). Then fog is a measurable function from (Qy, F1) 
to (Qs3, Fs). a 


3.1.5 Induced Measures and Induced Measure Spaces 


Proposition 3.1.6 Let f be a measurable function from (Q,F) to (Y,F'), and let wu be a 
measure on (Q,F). Let yu’ be defined on F' as follows: for each element C of F', w'(C) = 
u(f-'(C)). Then p! is a measure on (Q', F'). The measure pi! is called the measure induced by 
f, and is denoted by f(t). a 


Based on the result above, it is possible to transform a measure space using a function /f. 
Let (Q,F, 1) be a measure space, and let f be a function defined on Q. Let 9’ be f(Q), and 
let F’ be the set of subsets C of 0’ such that f-'(C) € F. Then, F’ is a o-field, and f is a 
measurable function from (2, F) to (Q’, F’). Thus, the space (2’, F’, f(j)) is a measure space. 
We call such a space the space induced by f, and we denote it by f((Q,F,)). Observe that 
if (Q,F, 4) is a probability space, then f((Q,F,)) is a probability space as well, and that 
induced measure spaces preserve discreteness and completeness. 


3.1.6 Product of Measure Spaces 


Let (Q1, 71) and (Q2, Fz) be two measurable spaces. Denote by F, ® F2 the o-field generated 
by the set of rectangles {Cy x Cz | C1 € Fi,Co € Fo}. The product space of (Q1, 71) and 
(Qe, F2), denoted by (Q1, Fi) @ (Qe, F2), is the measurable space (Qy x Q2, Fi ® F2). 


Proposition 3.1.7 Let (Q1, Fi, 1) and (Q2, F2, 2) be two measure spaces where ju, and j12 
are finite measures. Then there is a unique measure, denoted by 41 ®@ fo, on Fy ® Fa such that 


for each Cy € Fy and Cg € Fa, La ® Ho(Cy x C2) = Ha(C1)pla(C2). | 


The product measure space of two measure spaces (04, Fi, p41) and (Q2, F2, 2), denoted by 
(Qi, Fi, pi) ® (Qa, Fa, pa), is the measure space (Qy x Q2, Fy ® Fa, jy ® pa). It is easy to check 
that if (Q1, Fi, p1) and (Q2, Fe, 42) are probability spaces, then their product is a probability 
space as well. 

The product of two measure spaces is invertible. Let (Q, F, «) = (Qi, Fi, pr) @ (Qa, Fa, p2), 
and let 7;, 7 = 1,2, be a projection function from 1 x Q2 to Q;, that maps each pair (21, #2) 
to a;. Let Qf = 7;(Q;), and let F! = {C | m7 1(C) € Fi}. Then (0, F!) = (0;, F;), and 7; is 
a measurable function from (Q, 7) to (Q/, F/). The measure 7;(j:) coincides with ju;, since for 
each C € Fy, my '(C) = C x Qe, and for each C € Fo, ry'(C) = x C. Thus, the projection 
of (Q, F, 44) onto its i*® component is (Q;, F;, 14;). 


3.1.7 Combination of Discrete Probability Spaces 


In our theory there are several situations in which a discrete probability space is chosen accord- 
ing to some probability distribution, and then an element from the chosen probability space 
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is chosen according to the corresponding probability distribution. The whole process can be 
described by a unique probability space. 

Let {(Q;, Fi, Pi) }ioo be a family of discrete probability spaces, and let {p;}i>0 be a family 
of real numbers between 0 and | such that 37:5) p; = 1. Define >;5.9(Q:, Fi, P;) to be the triple 
(Q,F, P), where Q = Uj>0Q;, F = 2° and, for each a € Q, Pla] = Visowen, pi Pa]. It is easy 
to verify that (Q,F, P) is a probability space. 7 

The process described by (Q,F, P) is the following: a probability space (Q;, F;, P;) is drawn 
from {(Q;,F;, P;)}is0 with probability p;, and then an element a is drawn drom Q; with prob- 
ability P;[a]. 


3.1.8 Conditional Probability 


Let (Q,F, P) be a probability space, and let F be an element of F. Frequently, we need to 
study the probability of an event /’ of F knowing that event F’ has occurred. For example, we 
may want to study the probability that a dice rolled 6 knowing that it rolled a number greater 
than 3. The probability of a conditional event is expressed by P[E’|E]. If P[E] = 0, then 
P(E"|E] is undefined; if P[E] > 0, then P[E’|F] is defined to be P[E 0 E’|/P[E}. 

Suppose that P|E] > 0, and consider the triple (Q|F,F|E, P|E) where Q|E = BF, F|E = 
{E'N E | E' € F}, and for each event E’ of F|E, P|E[E") = P[E’|E]. Then it is easy to show 
that (Q|E, F|E, P|E) is a probability space. We call this space a conditional probability space. 

Conditional measures give us an alternative way to express the probability of the intersection 
of several events. That is, 


PLE A+++ E,] = PLE1]P[Eo| Ei] ---P[E,|E1 +++ Ena]: 


If P/E") = P[E"|E], then P[E 9 E"| = P[E|P[E"’. In this case the events F and EF’ are said 
to be independent. 


3.1.9 Expected Values 


Let (Q,F) be a measurable space, and let (R,R) be the measurable space where ¥ is the set 
of real numbers, and F® is the o-field generated by the open sets of the real line. A random 
variable on (Q, F), denoted by X, is a measurable function from (Q, F) to (R, R). 

We use random variables to deal with timed systems. An example of a random variable is 
the function that, given a computation of a system, returns the time it takes to the system to 
achieve a goal in the given computation. In our case, the computations of a system are chosen 
at random, and thus, a natural estimate of the performance of the system is the average time 
it takes to the system to achieve the given goal. 

The above idea is expressed formally by the expected value of a random variable, which is a 
weighted average of X. Specifically, let (Q, 7, P) be a probability space, and let XY be a random 
variable on (Q,F). Then the expected value of X, denoted by E[X], is the weighted average 
of X based on the probability distribution P. We do not show how to compute the expected 
value of a random variable in general, and we refer the interested reader to [Hal50]. Here we 
just mention that if Q can be partitioned in a countable collection of measurable sets (Cj)j>, 
such that for each set C;, X(C;) is a singleton, then E[X] = 37:39 P[C;|X(c;), where for each i 
c; is an element of F;. 7 
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3.1.10 Notation 


Throughout the thesis we adopt some conventional notation concerning probability spaces. We 
use the notation P, possibly decorated with indexes and primes, to denote a generic probability 
space. Thus, the expression P/ stands for the probability space (Q',F!, P/). Furthermore, if 
a generic expression ezp denotes a probability space (Q,7,P), we use Qeep, Ferp, and Pex to 
denote 2,7, and P, respectively. 

If (Q,F, P) is a probability space, and F is a generic set, we use P[E] to denote PLE AQ]. 
If £1Q is not an element of F, then P[F] is undefined. 

A special kind of probability space is a probability space with a unique element in its sample 
set. The corresponding measure is called a Dirac distribution. We use the notation D(a) to 
denote a probability space (2,7, P) where Q = {cz}. 

Another important kind of probability space is a space with finitely many elements, each 
one with the same probability. The corresponding measure is called a uniform distribution. 
We use the notation U/(a1,...,2,) to denote a discrete probability space (2,7, P) where Q = 
{z1,...,%,} and, for each element 2; of 2, Pla;] = 1/n. 

In the thesis we use heavily discrete probability spaces with no 0-probability elements. It 
is easy to verify that the sample set of these probability spaces is at most countable. If C' is 
any set, then we denote by Probs(C) the set of discrete probability spaces (Q,F,P) with no 
0-probability elements such that QO CC. 


3.2. Labeled Transition Systems 


A Labeled Transition System [Kel76, Plo81] is a state machine with labeled transitions. The 
labels, also called actions, are used to model communication between a system and its external 
environment. Labeled transition systems have been used successfully for the analysis of con- 
current and distributed systems [DH84, Mil89, LT87, LV93a]; for this reason we choose them 
as our basic model. 

Currently there are several definitions of labeled transition systems, each one best suited 
for the kind of application it is meant for. In this section we present a definition of labeled 
transition systems in the style of [LV93a]. 


3.2.1 Automata 
An automaton A consists of four components: 
1. a set states(A) of states. 
2. a nonempty set start(A) C states(A) of start states. 


3. an action signature stg(A) = (ezt(A), int(A)), where ext(A) and int(A) are disjoint sets 
of external and internal actions, respectively. Denote by acts(A) the set eat(A) U int(A) 
of actions. 


4. a transition relation trans(A) C states( A) x acts( A) x states( A). The elements of trans(A) 
are referred to as transitions or steps. 
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extract(1) 
[$$ 


insert(1) 


Figure 3-1: The Buffer automaton. 


Thus, an automaton is a labeled transition system, possibly with multiple start states, whose 
actions are partitioned into external and internal actions. The external actions model com- 
munication with the external environment; the internal actions model internal communication, 
not visible from the external environment. 

We use s to denote a generic state, and a and b to denote a generic action. We also use 7 to 
denote a generic internal action. All our conventional symbols may be decorated with primes 
and indexes. We say that an action a is enabled from a state s in A if there exists a state s’ of 
A such that (s,a,s’) is a transition of A. 

A standard alternative notation for transitions is s + s’. This notation can be extended to 
finite sequences of actions as follows: s “5” s’ iff there exists a sequence of states s1,..., 5,1 
such that s —5 s; —% ---8,_1 —% s,. To abstract from internal computation, there is another 
standard notion of weak transition, denoted by s => s’. The action a must be external, and 
the meaning of s => s’ is that there are two finite sequences (31, 32 of internal actions such that 
g 8 6 As for ordinary transitions, weak transitions can be generalized to finite sequences 
of external actions. A special case is given by the empty sequence: s => s’ iff either s’ = s or 


there exists a finite sequence @ of internal actions such that s a gli 


Example 3.2.1 A classic example of an automaton is an unbounded ordered buffer that stores 
natural numbers (see Figure 3-1). An external user sends natural numbers to the buffer, and 
the buffer sends back to the external environment the ordered sequence of numbers it receives 
from the user. 

The automaton Buffer of Figure 3-1 can be described as follows. All the actions of Buffer 
are external and are of the form insert(i) and extract(i), where i is a natural number, i.e., the 
actions of Buffer are given by the infinite set Ujen {insert(?), extract(i)}. The states of Buffer 
are the finite sequences of natural numbers, and the start state of Buffer is the empty sequence. 
The actions of the form insert(i) are enabled from every state of Buffer, i.e., for each state 
s and each natural number 7 there is a transition (s, insert(2),is) in Buffer, where is denotes 
the sequence obtained by appending 7 to the left of s. The actions of the form extract(i) are 
enabled only from those states where 7 is the rightmost element in the corresponding sequence 
of numbers, i.e., for each state s and each natural number ? there is a transition (s?, extract(7), s) 
of Buffer. No other transitions are defined for Buffer. 

Observe that from every state of Buffer there are infinitely many actions enabled. The 
way to choose among those actions is not specified in Buffer. In other words, the choice of the 
transition to perform is nondeterministic. In this case the nondeterminism models the arbitrary 
behavior of the environment. 
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insert(i) extract(i) 


Figure 3-2: Concatenation of two buffers. 


The role of internal actions becomes clear when we concatenate two buffers as in Figure 3-2. 
The communication that occurs between the two buffers is internal in the sense that it does not 
affect directly the external environment. Another useful observation about the concatenation 
of the two buffers in Figure 3-2 is that nondeterminism expresses two different phenomena: the 
arbitrary behavior of the environment, and the arbitrary scheduling policy that can be adopted 
in choosing whether Buffer, or Buffer, performs the next transition. In general nondeterminism 
can express even a third phenomenon, namely, the fact that an arbitrary state can be reached 
after the occurrence of an action. Such a form of nondeterminism would arise if we assume that 
a buffer may lose data by failing to modify its state during an insertion operation. | 


3.2.2 Executions 


The evolution of an automaton can be described by means of its executions. An execution 
fragment a of an automaton A is a (finite or infinite) sequence of alternating states and actions 
starting with a state and, if the execution fragment is finite, ending in a state 


Q = 9815142592 °°° 


where for each 2, (5;, 4:41, Si41) is a transition of A. Thus, an execution fragment represents a 
possible way to resolve the nondeterminism in an automaton. 

Denote by fstate(a) the first state of a and, if a is finite, denote by Istate(a) the last state of 
a. Furthermore, denote by frag*(A) and frag(A) the sets of finite and all execution fragments 
of A, respectively. 

An execution is an execution fragment whose first state is a start state. Denote by exec*( A) 
and exec(A) the sets of finite and all execution of A, respectively. A state s of A is reachable if 
there exists a finite execution of A that ends in s. 

The length of an execution fragment a, denoted by |a|, is the number of actions that occur 
in a. If a is infinite, then |a| = oo. 


A finite execution fragment a, = s9@ ,5,---d,5, of A and an execution fragment ag = 
$nAn418n41°°: Of A can be concatenated. In this case the concatenation, written a ~ ag, is 
the execution fragment 890181 +++ @p)Sn4n41$n41°°°. If a = ay 7 ag, then we denote ag by ara, 


(read “a after ay”). 

Anexecution fragment a, of A is a prefiz of an execution fragment a2 of A, written ay < a, 
if either ay = ag or ay is finite and there exists an execution fragment aj of A such that 
a2 = a,~ a}. The execution fragment a} is also called a suffix of a2 and is denoted by agra. 
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3.2.3 Traces 


The executions of an automaton contain a lot of information that is irrelevant to the environ- 
ment, since the interaction between an automaton and its environment occurs through external 
actions only. The trace of an execution is the object that represents the actual interaction that 
occurs between an automaton and its environment during an execution. 

The trace of an execution (fragment) a of an automaton A, written trace 4(a), or just 
trace(a) when A is clear, is the list obtained by restricting a to the set of external actions of 
A, ie., trace(a) = a [ ext(A). We say that @ is a trace of an automaton A if there exists an 
execution a of A with trace(a) = 3. Denote by traces*(A) and traces(A) the sets of finite and 
all traces of A, respectively. Note, that a finite trace can be the trace of an infinite execution. 


3.2.4 Trace Semantics 


In [LV93a] automata are compared based on traces. Specifically, a preorder relation is defined 
between automata based on inclusion of their traces: 


A, Cr Ag iff traces( Ay) C traces( Az). 


The trace preorder can express a notion of implementation, usually referred to as a safe imple- 
mentation. That is, A,, the implementation, cannot do anything that is forbidden by Ag, the 
specification. For example, no implementation of the buffer of Figure 3-1 can return natural 
numbers that were never entered or natural numbers in the wrong order. 

Although the trace preorder is weak as a notion of implementation, and so finer relations 
could be more appropriate [DeN87, Gla90, Gla93], there are several situations where a trace 
based semantics is sufficient [LT87, Dil88, AL93, GSSL94]. The advantage of a trace based 
semantics is that it is easy to handle. 

In this thesis we concentrate mainly on trace based semantics; however, the techniques that 
we develop can be extended to other semantic notions as well. 


3.2.5 Parallel Composition 


Parallel composition is the operator on automata that identifies how automata communicate 
and synchronize. There are two main synchronization mechanisms for labeled transition sys- 
tems, better known as the CCS synchronization style [Mil89], and the CSP synchronization 
style [Hoa85]. In the CCS synchronization style the external actions are grouped in pairs of 
complementary actions; a synchronization occurs between two automata that perform comple- 
mentary actions, and becomes invisible to the external environment, i.e., a synchronization is 
an internal action. Unless specifically stated through an additional restriction operator, an 
automaton is allowed not to synchronize with another automaton even though a synchroniza- 
tion is possible. In the CSP synchronization style two automata must synchronize on their 
common actions and evolve independently on the others. Both in the CCS and CSP styles, 
communication is achieved through synchronization. 

In this thesis we adopt the CSP synchronization style, which is essentially the style adopted 
in [LT87, Dil88, LV93a]. A technical problem that arises in our framework is that automata 
may communicate through their internal actions, while internal actions are not supposed to be 
visible. To avoid these unwanted communications, we define a notion of compatibility between 
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automata. Two automata A, Ag are compatible iff int(A,)M acts(A2) = O and acts(A,)N 
The parallel composition of two compatible automata A,, A, denoted by Aj||Ag, is the 
automaton A such that 


1. states(A) = states(A,) x states( Ag). 
2. start(A) = start(Ay) x start(Ag). 
3. sig(A) = (ext(A1) U ext( Ag), int( 1) U int( Ag). 
4. ((81, 82), 4, (84, .85)) € trans( A) iff 


a) if a € acts(A,), then (s1, 4,5) € trans(A,), else 54 = 51, and 
( ) 9 9M OL ? 1 9 
(b) if a € acts( Ag), then (s2, a, 84) € trans( Ag), else 34 = S92. 


If two automata are incompatible and we want to compose them in parallel, the problem 
can be solved easily by renaming the internal actions of one of the automata. The renaming 
operation is simple: just rename each occurrence of each action in the action signature and the 
transition relation of the given argument automaton. At this point it is possible to understand 
how to build a system like the one described in Figure 3-2. Buffer, is obtained from Buffer by 
renaming the actions eztract(7) into T(2), and Buffer, is obtained from Buffer by renaming the 
actions insert(z) into r(i). Then, Buffer; and Buffer, are composed in parallel, and finally the 
actions T(?) are made internal. This last step is achieved through a Hide operation, whose only 
effect is to change the signature of an automaton. 

We conclude by presenting two important properties of parallel composition. The first 
property concerns projections of executions. Let A = Aj,||A2, and let (51,52) be a state of A. 
Let 7 be either 1 or 2. The projection of (51,52) onto A;, denoted by (51, 52)[Aj, is s;. Let 
Q = S89418,--- be an execution of A. The projection of a onto A;, denoted by a[ A; is the 
sequence obtained from a by projecting all the states onto A;, and by removing all the actions 
not in acts(A;) together with their subsequent states. 


Proposition 3.2.1 Let A = Aj||Ao, and let a be an execution of A. Then a[ A, is an execution 
of Ay and a[ Ag is an execution of Ag. | 


The projection of an execution of A onto one of the components A; is essentially the view of 

A; of the execution a. In other words the projection represents what A; does in order for A to 

produce a. Proposition 3.2.1 states that the view of A; is indeed something that A; can do. 
The second property concerns the trace preorder. 


Proposition 3.2.2 Let Ay Cr Aj. Then, for each Ag compatible with both A, and Aj, 
Ay||A2 Lr A} || Ae. | 


The property expressed in Proposition 3.2.2 is better known as substitutivity or compositionality. 


In other words Er is a precongruence with respect to parallel composition. Substitutivity is one 
of the most important properties that an implementation relation should satisfy. Informally, 
substitutivity says that an implementation A, of a system A} works correctly in any context 
where Aj works correctly. Substitutivity is also the key idea at the base of modular verification 
techniques. 


Al 
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Chapter 4 


Probabilistic Automata 


4.1 What we Need to Model 


Our main goal is to analyze objects that at any point can evolve according to a probability 
distribution. The simplest example of a random computation is the process of flipping a coin. 
Thus, a program may contain an instruction like 


vie flip 


whose meaning is to assign to « the result of a coin flip. From the state-machine point of view, 
the transition relation of the corresponding automaton should be specified by giving the states 
reachable after the coin flip, together with their probability. Thus, the coin flipping process 
can be represented by the labeled transition system of Figure 4-1. The edges joining two states 
are associated with an action and a weight, where the weight of an edge is the probability of 
choosing that specific edge. Thus, we require that for each state that has some outgoing edges, 
the sum of the weights of the outgoing edges is 1. 

However, we also need to deal with nondeterminism. Consider a more complicated process 
where a coin is flipped, but where the coin can be either fair, i.e., it yields head with probability 
1/2, or unfair by yielding head with probability 2/3. Furthermore, suppose that the process 
emits a beep if the result of the coin flip is head. In this case, the choice of which coin to flip 
is nondeterministic, while the outcome of the coin flip is probabilistic. The start state should 
enable two separate transitions, each one corresponding to the flip of a specific coin. Figure 4- 
2 represents the nondeterministic coin flipping process. The start state enables two separate 
groups of weighted edges; each group is identified by an arc joining all of its edges, and the 
edges of each group form a probability distribution. 

At this point we may be tempted to ask the following question: 


head 


tail 


Figure 4-1: The coin flipping process. 
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beep 


head ————~> s 


flip 


Figure 4-2: The nondeterministic coin flipping process. 


“What is the probability that the nondeterministic coin flipper beeps?” 
The correct answer is 
“Tt depends on which coin is flipped.” 


Although this observation may appear to be silly, the lesson that we learn is that it is not 
possible to talk about the probability of some event until the nondeterminism is resolved. 
Perhaps we could give a more accurate answer as follows: 


“The probability that the nondeterministic coin flipper beeps is either 1/2 or 2/3, 
depending on which coin is flipped.” 


However, there are two possible objections. The first objection concerns the way a coin is 
chosen. What happens if the coin to be flipped is chosen at random? After all, in the definition 
of the nondeterministic coin flipper there are no limitations to the way a coin is chosen. In this 
case, the correct answer would be 


“The probability that the nondeterministic coin flipper beeps is between 1/2 and 2/3, 
depending on how the coin to be flipped is chosen.” 


The second objection concerns the possibility of scheduling a transition. What happens if the 
scheduler does not schedule the beep transition even though it is enabled? In this case the 
correct answer would be 


“Under the hypothesis that some transition is scheduled whenever some transition is 
enabled, the probability that the nondeterministic coin flipper beeps is between 1/2 
and 2/3, depending on how the coin to be flipped is chosen.” 


There is also another statement that can be formulated in relation to the question: 


“The nondeterministic coin flipper does not beep with any probability greater than 
2/3." 


This last property is better known as a safety property [AS85] for ordinary labeled transition 
systems. 

Let us go back to the scheduling problem. There are actual cases where it is natural to allow 
a scheduler not to schedule any transition even though some transition is enabled. Consider a 
new nondeterministic coin flipper with two buttons, marked fair and unfair, respectively. The 
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Figure 4-3: The triggered coin flipping process. 
head beep 
tail 


beep 
head —————> 5 


tail 


Figure 4-4: A computation of the triggered coin flipping process. 


buttons can be pressed by an external user. Suppose that pressing one button disables the 
other button, and suppose that the fair coin is flipped if the button marked fair is pressed, 
and that the unfair coin is flipped if the button marked unfair is pressed. The new process 
is represented in Figure 4-3. In this case the scheduler models the external environment, and 
a user may decide not to press any button, thus not scheduling any transition from sg even 
though some transition is enabled. An external user may even decide to flip a coin and press 
a button only if the coin gives head, or flip a coin and press fair if the coin gives head and 
press unfair if the coin gives tail. That is, an external user acts like a scheduler that can use 
randomization for its choices. If we ask again the question about the probability of beeping, a 
correct answer would be 


“Assuming that beep is scheduled whenever it is enabled, the probability that the 
triggered coin flipper beeps, conditional to the occurrence of a coin flip, is between 
L/2and 2/3,” 


Suppose now that we resolve all the nondeterminism in the triggered coin flipper of Figure 4-3, 
and consider the case where the external user presses fair with probability 1/2 and unfair 
with probability 1/2. In this case it is possible to study the exact probability that the process 
beeps, which is 7/12. Figure 4-4 gives a representation of the outcome of the user we have just 
described. Note that the result of resolving the nondeterminism is not a linear structure as is 
the case for standard automata, but rather a tree-like structure. This structure is our notion 
of a probabilistic execution and is studied in more detail in Section 4.2. 
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4.2 The Basic Model 


In this section we introduce the basic probabilistic model that is used in the thesis. We formalize 
the informal ideas presented in Section 4.1, and we extend the parallel composition operator 
of ordinary automata to the new framework. We also introduce several notational conventions 
that are used throughout the thesis. 


4.2.1 Probabilistic Automata 


A probabilistic automaton M consists of four components: 
1. A set states(M) of states. 
2. A nonempty set start(M) C states(M) of start states. 


3. An action signature sig(.M) = (ext(M), int(M)), where ext(.M) and int(M) are disjoint 
sets of external and internal actions, respectively. Denote by acts(M) the set ext(M) U 
int(.M) of actions. 


4. A transition relation trans(.M) C states(.M) x Probs((acts(.M) x states(M))U{6}). Recall 
from Section 3.1.10 that for each set C’, Probs(C’) denotes the set of discrete probability 
spaces (Q,F,P) with no 0-probability elements such that Q C C. The elements of 
trans(.M) are referred to as transitions or steps. 


A probabilistic automaton differs from an ordinary automaton only in the transition relation. 
Each transition represents what in the figures of Section 4.1 is represented by a group of edges 
joined by an arc. From each state s, once a transition is chosen nondeterministically, the 
action that is performed and the state that is reached are determined by a discrete probability 
distribution. Each transition (s,P) may contain a special symbol 6, which represents the 
possibility for the system not to complete the transition, i.e., to remain in s without being able 
to engage in any other transition. 


Example 4.2.1 (Meaning of 6) To give an idea of the meaning of 6, suppose that M models 
a person sitting on a chair that stands up with probability 1/2. That is, from the start state so 
there is a transition of Mf where one outcome describes the fact that the person stands up and 
the other outcome describes the fact that the person does not stand up (this is 6). The point 
is that there is no instant in time where the person decides not to stand up: there are only 
instants where the person stands up. What the transition leaving 59 represents is that overall 
the probability that the person does the action of standing up is 1/2. The need for 6 is clarified 
further in Section 4.2.3, where we study probabilistic executions, and in Section 4.3, where we 
study parallel composition. | 


The requirement that the probability space associated with a transition be discrete is imposed 
to simplify the measure theoretical analysis of probabilistic automata. In this thesis we work 
with discrete probability spaces only, and we defer to further work the extension of the theory 
to more general probability spaces. The requirement that each transition does not lead to any 
place with probability 0 is imposed to simplify the analysis of probabilistic automata. All the 
results of this thesis would be valid even without such a restriction, although the proofs would 
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contain a lot of uninteresting details. The requirement becomes necessary for the study of live 
probabilistic automata, which we do not study here. 

There are two classes of probabilistic automata that are especially important for our analysis: 
simple probabilistic automata, and fully probabilistic automata. 

A probabilistic automaton M is simple if for each transition (s,P) of trans(M) there is an 
action a of M such that Q C {a} x states(M). In such a case, a transition can be represented 
alternatively as (s,a,P"), where P’ € Probs(states(M)), and it is called a simple transition with 
action a. The probabilistic automata of Figures 4-2 and 4-3 are simple. In a simple probabilistic 
automaton each transition is associated with a single action and it always completes. The idea 
is that once a transition is chosen, then only the next state is chosen probabilistically. In 
this thesis we deal mainly with simple probabilistic automata for a reason that is made clear 
in Section 4.3. We use general probabilistic automata to analyze the computations of simple 
probabilistic automata. 

A probabilistic automaton M is fully probabilistic if M has a unique start state, and from 
each state of M there is at most one transition enabled. Thus, a fully probabilistic automaton 
does not contain any nondeterminism. Fully probabilistic automata play a crucial role in the 
definition of probabilistic executions. 


Example 4.2.2 (Probabilistic automata) A probabilistic Turing Machine is a Turing ma- 
chine with an additional random tape. The content of the random tape is instantiated by 
assigning each cell the result of an independent fair coin flip (say 0 if the coin gives head and 
1 if the coin gives tail). If we assume that each cell of the random tape is instantiated only 
when it is reached by the head of the machine, then a probabilistic Turing machine can be 
represented as a simple probabilistic automaton. The probabilistic automaton, denoted by M, 
has a unique internal action 7, and its states are the instantaneous descriptions of the given 
probabilistic Turing machine; each time the Turing machine moves the head of its random tape 
on acell for the first time, WV has a probabilistic transition that represents the result of reaching 
a cell whose content is 0 with probability 1/2 and 1 with probability 1/2. 

An algorithm that at some point can flip a coin or roll a dice can be represented as a simple 
probabilistic automaton where the flipping and rolling operations are simple transitions. If the 
outcome of a coin flip or dice roll affects the external behavior of the automaton, then the 
flip and roll actions can be followed by simple transitions whose actions represent the outcome 
of the random choice. Another possibility is to represent the outcome of the random choice 
directly in the transition where the random choice is made by performing different actions. In 
this case the resulting probabilistic automaton would not be simple. Later in the chapter we 
show why we prefer to represent systems as simple probabilistic automata when possible. 


4.2.2 Combined Transitions 


In Section 4.1 we argued that a scheduler may resolve the nondeterminism using randomization, 
i.e., a scheduler can generate a new transition by combining several transitions of a probabilistic 
automaton M. We call the result of the combination of several transitions a combined transition. 
Formally, let M be a probabilistic automaton, and let s be a state of M. Consider a finite or 
countable set {(s,P;)}ier of transitions of M leaving from s, and a family of non-negative 
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weights {p;}iez such that >; p; < 1. Let 


Ps Ss” piPi | + ( — rn) D(6), (4.1) 


i€I|p;>0 wel 


ie., P is a combination of discrete probability spaces as described in Section 3.1.7. The 
pair (s,P) is called a combined transition of M and is denoted by oj; pi(s, Pi). Denote 
by ctrans(M) the set of combined transitions of M. Note that trans(M) C ctrans(M). 

Thus, the combination of transitions can be viewed as a weighted sum of transitions where 
the sum of the weights is at most 1. If the sum of the weights is not 1, then nothing is 
scheduled by default. The reason for 6 by default will become clear when we analyze parallel 
composition in Section 4.3. Note that all the transitions (s,P;) where p; = 0 are discarded in 
Expression (4.1), since otherwise P would contain elements whose probability is 0. We do not 
impose the restriction that each p; is not 0 for notational convenience: in several parts of the 
thesis the p;’s are given by complex expression that sometimes may evaluate to 0. 


Proposition 4.2.1 The combination of combined transitions of a probabilistic automaton M 
is a combined transition of M. 


Proof. Follows trivially from the definition of a combined transition. | 


4.2.3. Probabilistic Executions 


If we resolve both the nondeterministic and probabilistic choices of a probabilistic automaton, 
then we obtain an ordinary execution like those usually defined for ordinary automata. Thus, an 
execution fragment of a probabilistic automaton M is a (finite or infinite) sequence of alternating 
states and actions starting with a state and, if the execution fragment is finite, ending in a state, 


QO = $941814252°°°, 


where for each 7 there is a transition (s;,Pi41) of M such that (aj41, 5:41) € Qi41. Executions, 
concatenations of executions, and prefixes can be defined as for ordinary automata. 

In order to study the probabilistic behavior of a probabilistic automaton, we need a mech- 
anism to resolve only the nondeterminism, and leave the rest unchanged. That is, we need a 
structure that describes the result of choosing a transition, possibly using randomization, at 
any point in history, i.e., at any point during a computation. In Figure 4-4 we have given an 
example of such a structure, and we have claimed that it should look like a tree. Here we give 
a more significant example to justify such a claim. 


Example 4.2.3 (History in a probabilistic execution) Consider a new triggered coin flip- 
per, described in Figure 4-5, that can decide nondeterministically to beep or boo if the coin flip 
yields head, and consider a computation, described in Figure 4-6, that beeps if the user chooses 
to flip the fair coin, and boos if the user chooses to flip the unfair coin. Then, it is evident that 
we cannot identify the two states head of Figure 4-6 without reintroducing nondeterminism. In 
other words, the transition that is scheduled at each point depends on the past history of the 
system, which is represented by the position of a state in the tree. For a formal definition of a 
structure like the one of Figure 4-6, however, we need to refer explicitly to the past history of 
a system. a 
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boo 


tail 


Figure 4-5: The triggered coin flipper with a boo sound. 
head td 
tail 


boo 


head —————_=> s' 


tail 


Figure 4-6: A computation of the triggered coin flipper with a boo sound. 


Let a be a finite execution fragment of a probabilistic automaton M. Define a function a~ 
that applied to a pair (a,s) returns the pair (a,aas), and applied to 6 returns 6. Recall from 
Section 3.1.5 that the function a~ can be extended to probability spaces. Informally, if (s,P) is 
a combined transition of M and a is a finite execution fragment of M such that Istate(a) = s, 
then the pair (a,a~ P) denotes a transition of a structure that in its states remembers part of 
the past history. A probabilistic execution fragment of a probabilistic automaton M, is a fully 
probabilistic automaton, denoted by A, such that 


1. states( H) C frag*(M). Let q range over states of probabilistic execution fragments. 


2. for each transition (q,P) of H there is a combined transition (Istate(q),P’) of M, called 
the corresponding combined transition, such that P = q~ P’. 


3. each state g of H is reachable in H and enables one transition, possibly (¢, D(é)). 


A probabilistic execution is a probabilistic execution fragment whose start state is a start state of 
M. Denote by prfrag(M) the set of probabilistic execution fragments of M, and by prerec( M) 
the set of probabilistic executions of M. Also, denote by qé! the start state of a generic 
probabilistic execution fragment H. 

Thus, by definition, a probabilistic execution fragment is a probabilistic automaton itself. 
Condition 3 is technical: reachability is imposed to avoid useless states in a probabilistic exe- 
cution fragment; the fact that each state enables one transition is imposed to treat uniformly 
all the points where it is possible not to schedule anything. Figures 4-6 and 4-7 represent 
two probabilistic executions of the triggered coin flipper of Figure 4-5. The occurrence of 6 
is represented by a dashed line labeled with 6. The states of the probabilistic executions are 
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Figure 4-7: A probabilistic execution of the triggered coin flipper. 


not represented as finite execution fragments since their position in the diagrams gives enough 
information. Similarly, we omit writing explicitly all the transitions that lead to D(6) (e.g., 
states sy and sz in Figure 4-7). 

We now have enough structure to understand better the role of 6. In ordinary automata a 
scheduler has the possibility not to schedule anything at any point, leading to a finite execution. 
Such assumption is meaningful if the actions enabled from a given state model some input 
that comes from the external environment. In the probabilistic framework it is also possible to 
schedule no transition from some point. Since a scheduler may use randomization in its choices, 
it is also possible that from some specific state nothing is scheduled only with some probability 


p, say 1/2. 


Example 4.2.4 (The role of 6) In the triggered coin flipper of Figure 4-5 a user can flip 
a fair coin to decide whether to push a button, and then, if the coin flip yields head, flip 
another coin to decide which button to press. In the transition that leaves from sg we need 
some structure that represents the fact that nothing is scheduled from s9 with probability 1/2: 
we use 6 for this purpose. Figure 4-7 represents the probabilistic execution that we have just 
described. | 


Since a probabilistic execution fragment is itself a probabilistic automaton, it is possible to 
talk about the executions of a probabilistic execution fragment, that is, the ways in which the 
probabilistic choices can be resolved in a probabilistic execution fragment. However, since at 
any point gq it is possible not to schedule anything, if we want to be able to study the probabilistic 
behavior of a probabilistic execution fragment then we need to distinguish between being in g¢ 
with the possibility to proceed and being in g without any possibility to proceed. For example, 
in the probabilistic execution of Figure 4-7 we need to distinguish between being in sq before 
performing the transition enabled from sg and being in so after performing the transition. We 
represent this second condition by writing s9é. In general, we introduce a notion of an extended 
execution fragment, which is used in Section 4.2.5 to study the probability space associated with 
a probabilistic execution. 

An extended execution (fragment) of a probabilistic automaton M, denoted by a, is either 
an execution (fragment) of M, or a sequence a’é, where a’ is a finite execution (fragment) of 
M. The sequences 596 and so fair 5,6 are examples of extended executions of the probabilistic 
execution of Figure 4-7. 

There is a close relationship between the extended executions of a probabilistic automaton 
and the extended executions of one of its probabilistic execution fragments. Here we define 
two operators that make such a relationship explicit. Let M7 be a probabilistic automaton and 
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let H be a probabilistic execution fragment of M. Let go be the start state of H. For each 
extended execution @ = goa ,q, --- of A, let 


ol A qo ~ lstate( qo )aylstate(q )az--- if a@ does not end in 6, (4.2) 
~ qo ~ lstate( qo )aylstate(q )a2---a,lstate(q,)6 if a = qodiqi +++ Angnd. . 
It is immediate to observe that a] is an extended execution fragment of M. For each extended 
execution fragment a of M such that qo < a, i.e., @ = qo 594151 ---, let 
A go1( qo ~ 804151 )A2(qo ~ 8941514282) +-- if a@ does not end in 6, 
algo = 7 ek (4.3) 
goa1(4o $9 a1 51) . -(o $0415, °° An Sn, )6 if @ = Go ~ 804181 +++ An Sp 6. 


It is immediate to observe that afgo is an extended execution of some probabilistic execution 
fragment of Mf. Moreover, the following proposition holds. 


Proposition 4.2.2 Let H be a probabilistic execution fragment of a probabilistic automaton 
M, and let qo be the start state of H. Then, for each extended execution a of H, 


(|) qo =a, (4.4) 
and for each extended execution fragment a of M starting with qo, 

(algo)! =a. (4.5) 
Proof. Simple analysis of the definitions. | 


The bottom line is that it is possible to talk about extended executions of H by analyzing only 
extended execution fragments of M. 


4.2.4 Notational Conventions 


For the analysis of probabilistic automata and of probabilistic executions we need to refer to 
explicit objects like transitions or probability spaces associated with transitions. In this section 
we give a collection of notational conventions that ease the identification of each object. 


Transitions 


We denote a generic transition of a probabilistic automaton by tr, possibly decorated with 
primes and indices. For each transition tr = (s,P), we denote P alternatively by P;,. If tr isa 
simple transition, represented by (s,a,P), we abuse notation by denoting P by P;, as well. The 
context will always clarify the probability space that we denote. If (s,P) is a transition, we use 
any set of actions V to denote the event {(a,s’) € 2|a€ V} that expresses the occurrence of 
an action from V in P, and we use any set of states U to denote the event {(a,s’) EQ | s’ ce U} 
that expresses the occurrence of a state from U in P. We drop the set notation for singletons. 
Thus, P[a] is the probability that action a occurs in the transition (s,P). 

If M is a fully probabilistic automaton and s is a state of M, then we denote the unique 
transition enabled from s in M by tr™, and we denote the probability space that appears in 
tr’ by PM. Thus, tr’ = (s,P/). We drop M from the notation whenever it is clear from 
the context. This notation is important to handle probabilistic execution fragments. 
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Transition Prefixing and Suffixing 


Throughout the thesis we use transitions of probabilistic automata and transitions of proba- 
bilistic execution fragments interchangeably. If H is a probabilistic execution fragment of a 
probabilistic automaton M, then there is a strong relation between the transitions of H and 
some of the combined transitions of 17. We exploit such a correspondence through two oper- 
ations on transitions. The first operation is called transition prefixing and adds some partial 
history to the states of a transition; the second operation is called transition suffixing and re- 
moves some partial history from the states of a transition. These operations are used mainly 
in the proofs of the results of this thesis. 

Let tr = (s,P) be a combined transition of a probabilistic automaton M, and let a be a 
finite execution fragment of M such that Istate(a) = s. Then the transition a~ tr is defined to 
be (a,a~™ P). We call the operation a@ transition prefixing. 

Let tr = (q,P) be a transition of a probabilistic execution fragment H, and let q’ < q. Let 
pq’ be a function that applied to a pair (a,q") of Q returns (a, eq’), and applied to 6 returns 
6. Let Peg’ denote the result of applying eq’ to P. Then the transition tre’ is defined to be 
(qog’, Peg’). We call the operation cq’ transition suffixing. 

The following properties concern distributivity of transition prefixing and suffixing with 
respect to combination of transitions. 


Proposition 4.2.3 Let M be a probabilistic automaton, and let q be a finite execution fragment 


of M. 
1. ¢@7 >; pitri = 2; pilq> tri), where each tr; is a transition of M. 


2. 0, pitriog = 2; piltrieg), where each tr; is a transition of some probabilistic execution 
fragment of M. 


Proof. Simple manipulation of the definitions. | 


4.2.5 Events 


At this point we need to define formally how to compute the probability of some event in 
a probabilistic execution. Although it is intuitively simple to understand the probability of 
a finite execution to occur, it is not as intuitive to understand how to deal with arbitrary 
properties. A probabilistic execution can be countably branching, and can have uncountably 
many executions. As an example, consider a probabilistic execution that at any point draws a 
natural number n > 0 with probability 1/2”. What is measurable? What is the probability of 
a generic event? 

In this section we define a suitable probability space for a generic probabilistic execution 
fragment H of a probabilistic automaton M. Specifically, given a probabilistic execution frag- 
ment H we define a probability space Py as the completion of another probability space Pj, 
which is defined as follows. Define an extended execution a of H to be complete iff either a 
is infinite or a = a’6 and 6 € QH Then, the sample space 94, is the set of extended 


istate(a’)” 
executions of MW that originate from complete extended executions of H, i.e., 


Q', = {a| | a is a complete extended execution of H}. (4.6) 
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The occurrence of a finite extended execution a of M can be expressed by the set 
CH = {a EN, | a <a’'}, (4.7) 


called a cone. We drop H from C# whenever it is clear from the context. Let Cy be the set of 
cones of H. Then define Fj, to be the o-field generated by Cy, i-e., 


To define a probability measure on Fj, we start by defining a measure py on Cy such that 
y(Qy) = 1. Then we show that wy can be extended uniquely to a measure jig on F(Cy), 
where F(Cj7) is built according to Proposition 3.1.1. Finally we use the extension theorem 
(Theorem 3.1.2) to show that jy can be extended uniquely to a probability measure Py, on 
o(f'(Cy)) = o(Cy). 

The measure puy(C) of a cone C# is the product of the probabilities associated with each 
edge that generates a in H. Formally, let go be the start state of H. If a < qo, then 


p(C) = 1; (4.9) 
if @ = qo ~ $041 81 +++ Sn_—14nSn, then 


pH(C2) = Para] PH [ans Ind], (4.10) 


> 


where for each 2, 1 <2 < n, qj = go™ 800181 +--+ 814; 5;; If @ = qo ~ S941 81° ++ Sn_147576, then 


wa(C2) = PA lar,a)) PH [ans In) Pan lS], (4.11) 


where for each 2, 1 <2 <n, qj = Go~ 800181 ++ * 81 4j5;. 


Example 4.2.5 (Some commonly used events) Before proving that the construction of 
Pi, is correct, we give some examples of events. The set describing the occurrence of an action 
a (eventually @ occurs) can be expressed as a union of cones of the form C’, such that a appears 
in a. Moreover, any union of cones can be described as a union of disjoint cones (follows from 
Lemma 4.2.4 below). Since a probabilistic execution fragment is at most countably branching, 
the number of distinct cones in Cy is at most countable, and thus the occurrence of a can be 
expressed as a countable union of disjoint cones, i.e., it is an event of Fj,. More generally, any 
arbitrary union of cones is an event. We call such events finitely satisfiable. The reason for the 
word “satisfiable” is that it is possible to determine whether an execution a of 04, is within a 
finitely satisfiable event by observing just a finite prefix of a. That finite prefix is sufficient to 
determine that the property represented by the given event is satisfied. 

The set describing the non-occurrence of an action a is also an event, since it is the comple- 
ment of a finitely satisfiable event. Similarly, the occurrence, or non-occurrence, of any finite 
sequence of actions is an event. For each natural number n, the occurrence of exactly n a’s is 
an event: it is the intersection of the event expressing the occurrence of at least n a’s and the 
event expressing the non-occurrence of n+ 1 a’s. Finally, the occurrence of infinitely many a’s 
is an event: it is the countable intersection of the events expressing the occurrence of at least 7 
a’s,ti>0. | 
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We now move to the proof that Pj, is well defined. First we use ordinal induction to show that 
the function jy defined on Cy is o-additive, and thus that jy is a measure on Cy (Lemma 4.2.6); 
then we show that there is a unique extension of jy to (Cy) (Lemmas 4.2.7, 4.2.8, and 4.2.9). 
Finally, we use the extension theorem to conclude that Py, is well defined. 


Lemma 4.2.4 Let Co,,Co, € QH. If ay < ag then Ca, C Co,. Ifay ¢ a2 and ag ¢ a, then 
Ca, Co, = 9. 


Proof. Simple analysis of the definitions. | 


Lemma 4.2.5 Let H be a probabilistic execution of a probabilistic automaton M, and let q be 
a state of H. Suppose that there is a transition enabled from q in H. Then 


V(ag)ent HH(C4) if 6 € Qi 

MH(Cy) = (eaveng c. C f 6 Qt (4.12) 
Dagyent Ha(Cy) + par(Cgs) if 6 € QS". 

Proof. Simple analysis of the definitions. | 


Lemma 4.2.6 The function wy is o-additive on Cy, and wy(Qy) = 1. 


Proof. By definition wy(Q5,) = 1, hence it is sufficient to show o-additivity. Let ¢ be an 
extended execution of M, and let © be a set of incomparable extended executions of M such 
that Cy = UgeoCy. If q ends in 6, then © contains only one element and o-additivity is 
trivially satisfied. Thus, assume that g does not end in 6, and hence q is a state of H, and that 
© contains at least two elements. From Lemma 4.2.4, q is a prefix of each extended execution 
of ©. For each state q’ of H, let O, be the set {¢” € O | qd < q’}. We show o-additivity 
in two steps: first we assign an ordinal depth to some of the states of H and we show that q 
is assigned a depth; then we show that wa(C,) = Y4co MH(Cy’) by ordinal induction on the 
depth assigned to gq. 

The depth of each state q within some cone Cyn (q” < q’), where q” € O, is 0, and the depth 
of each state g’ with no successors is 0. For each other state q’ such that each of its successors 
has a depth, if {depth(q”) | du(a,¢) € any has a maximum, then 


depth(q') = mazx({depth(q") | Jala, ¢”) € OF) +1, (4.13) 


otherwise, if {depth(q”) | da(a,q”) € Qqr} does not have a maximum, then 


depth(q’) = sup({depth(q") | da(a,q") € QF 4). (4.14) 


Consider a maximal assignment to the states of H,i.e., an assignment that cannot be extended 
using the rules above, and suppose by contradiction that q is not assigned a depth. Then 
consider the following sequence of states of H. Let qo = q, and, for each 2 > 0, let q; be a state 
of H such that (a;,q;) € Qg,_,, and q; is not assigned a depth. For each 2, the state q; exists 
since otherwise, if there exists an 7 such that for each (a;,q@;) € Qg;_,, G@ is assigned a depth, 
then qg;-1 would be assigned a depth. Note that the g;’s form a chain under prefix ordering, i.e., 
for each 2,7, if 7 <7 then q < q;. Consider the execution a. = lim; q;. From its definition, ag, 
is an execution of Cj. Then, from hypothesis, a is an execution of Ugre@Cy, and therefore 
Qo is an execution of some Cy, such that q/ € ©. By definition of a cone, q’ is a prefix of a. 
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Thus, q’ = q, for some k > 0. But then gq, is within the cone Cy, and thus it is assigned depth 
0. This contradicts the fact that q; is not assigned any depth. 

Let 7 be the ordinal depth assigned to q. We show that wy(Cy) = Lycee MA(Cy) by 
ordinal induction on 7. If y = 0, then O is either {q} or {qé}, and the result is trivial. Let 
7 be a successor ordinal or a limit ordinal. From Lemma 4.2.5, ty(Cq) = (agen, HH(Cy) 
if 6 € Qq, and pa(Cy) = Veagyea, HA(Cq') + MA(Cys) if 6 € Qg. For each (a,q’) € Qy, 
Cy = Ugreo Cy" Moreover, for each (a,q’) € Q,, the depth of q’ is less than y. By induction, 
by(Cy) = Ligle®,) ty(Cqv). Thus, fa (Cy) = di (a,q")EQ, Liglee,) MH(Cqr) = diglee My(Cy) if 
6 €Qq, and py(Cy) = Di (aq)EQ, Liglee,) MH(Cqr) + fa (Cys) = Vgreo HH(Ca) if 6 € Qa. a 


Lemma 4.2.7 There exists a unique extension pi, of ur to F\(Cy). 


Proof. There is a unique way to extend the measure of the cones to their complements since 
for each a, py(Co) + wH(QH — Ca) = 1. Therefore y5, coincides with 7 on the cones and 
is defined to be 1 — py(C.) for the complement of any cone C,. Since, by the countably 
branching structure of H, the complement of a cone is a countable union of cones, o-additivity 
is preserved. | 


Lemma 4.2.8 There exists a unique extension pt, of jy, to Fo(Cy). 


Proof. The intersection of finitely many sets of F)(C#) is a countable union of cones. Therefore 
o-additivity enforces a unique measure on the new sets of F) (Cy). a 


Lemma 4.2.9 There exists a unique extension yt, of wii, to F3(Cy). 


Proof. There is a unique way of assigning a measure to the finite union of disjoint sets whose 
measure is known, i.e., adding up their measures. Since all the sets of /3(Cy) are countable 
unions of cones, o-additivity is preserved. | 


Theorem 4.2.10 There exists a unique extension Pi of uy to the o-algebra Fy. 
Proof. By Theorem 3.1.2, define P, to be the unique extension of ju; to Fi,. a 


4.2.6 Finite Probabilistic Executions, Prefixes, Conditionals, and Suffixes 


We extend the notions of finiteness, prefix and suffix to the probabilistic framework. Here we 
add also a notion of conditional probabilistic execution which is not meaningful in the non- 
probabilistic case and which plays a crucial role in some of the proofs of Chapter 5. 


Finite Probabilistic Executions 


Informally, finiteness means that the tree representation of a probabilistic execution fragment 
has a finite depth. Thus, a probabilistic execution fragment H is finite iff there exists a natural 
number n such that the length of each state of H is at most n. 
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Figure 4-8: Examples of the prefix relation. 


Prefixes 


The idea of a prefix of a probabilistic execution fragment is more complicated than the definition 
of prefix for ordinary automata. To get a better understanding of the problem, consider the 
definition of prefix for ordinary execution fragments: a < a’ iff either a = a’, or a is finite and 
there is an execution fragment a” such that a’ = a7 a”. Another way to interpret this definition 
is to observe that if a is finite, then there is exactly one point in a, which we call a point of 
extension, from which nothing is scheduled, and in that case a’ is obtained by extending a from 
its unique point of extension. With the word “extending” we mean “adding transitions”. In 
other words, an execution fragment a is a prefix of an execution fragment a’ iff a’ is obtained 
from a by adding transitions, possibly none, from all the points of extension of a, i.e., from 
all the points of a where nothing is scheduled. We apply the same observation to probabilistic 
execution fragments, where a point of extension is any point where 6 occurs. 


Example 4.2.6 (Prefixes) Consider the probabilistic execution fragment H of Figure 4-8. 
It is easy to see that s; and sg are points of extension in H. However, also so is a point 
of extension since in H nothing is scheduled from s9 with probability 1/2. The probabilistic 
execution fragment H’ of Figure 4-8 is an extension of H. States s; and sg are extended with 
transitions labeled with c, and half of the extendible part of so is extended with the transition 
80 —> 81, ie., we have added the transition (s9,2/((a, 1), 6)) to the extendible part of so. Since 
the extension from sg overlaps with one of the edges leaving sp in A, the effect that we observe 
in H’ is that s, is reached with a higher probability. 

Consider now the probabilistic execution fragment H” of Figure 4-8. H” is an extension 
of H’, but this time something counterintuitive has happened; namely, the edge labeled with 
action ¢ that leaves from state sy has a lower probability in H” than in H’. The reason for this 


difference is that the extendible part of so is extended with a transition so ey 8 followed by 
sy —> s’. Thus, half of the transition leaving from sj in H” is due to the previous behavior of 
Hf’, and half of the transition leaving from s2 in H” is due to the extension from 59. However, 
the probability of the cone C’,5s,cs is the same in H’ and in H”. | 


A formal definition of a prefix works as follows. A probabilistic execution fragment # is a prefix 
of a probabilistic execution fragment H’, denoted by H < H’, iff 


1. H and H’ have the same start state, and 


2. for each state g of H, Py[Cy] < Pa [Cy]. 


Observe that the definition of a prefix for ordinary executions is a special case of the definition 
we have just given. 
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Figure 4-9: Conditionals and suffixes. 


Conditionals 


Let H be a probabilistic execution fragment of a probabilistic automaton M, and let q be either 
a state of H or a prefix of the start state of H. We want to identify the part of H that describes 
what happens conditional to the occurrence of g. The new structure, which we denote by H|q, 
is a new probabilistic execution fragment defined as follows: 


1. states(H\|q) = {¢' € states(H) | q < q'}; 


2. start( H|q) = min(states( H|q)), where the minimum is taken under prefix ordering, 


Hild _ pH 


3. for each state q’ of H|q, try, = tr}. 


H\q is called a conditional probabilistic execution fragment. 


Example 4.2.7 (Conditionals) The probabilistic execution fragment H, of Figure 4-9 is an 
example of a conditional probabilistic execution fragment. Specifically, H; = H”|(soas2), where 
Hf" is represented in Figure 4-8. In Figure 4-9 we represent explicitly the states of A for clarity. 
The conditional operation essentially extracts the subtree of H” that starts with soas2. | 


It is easy to check that (Qr)9,Fa|q, Pq) and (Qu|Cy, Fu|Cy, Pu|Cq) are the same probability 
space (cf. Section 3.1.8). Indeed, the sample sets are the same, the generators are the same, and 
the probability measures coincide on the generators. Thus, the following proposition, which is 
used in Chapter 5, is true. 


Proposition 4.2.11 Let H be a probabilistic execution fragment of a probabilistic automaton 
M, and let q be either a state of H, or a prefix of the start state of H. Then, for each subset 
£ of OQ Iq, 


1. E€ Fug iff E € Fy. 


2. If E is an event, then Py|E] = Pu[C,]Pryql]. = 


Suffixes 


The definition of a suffix is similar to the definition of a conditional; the difference is that in 
the definition of Hpq we drop qg from each state of H, i.e., we forget part of the past history. 
Formally, let H be a probabilistic execution fragment of a probabilistic automaton M, and let 
q be either a state of H or a prefix of the start state of H. Then Hog is a new probabilistic 
execution fragment defined as follows: 


1. states( Hog) = {q'>q | q' € states(H),q < '}, 
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2. start( H>q) = min(states( Heq)), where the minimum is taken under prefix ordering, 


3. for each state q/ of H’, ripe = wrt ed: 
Hog is called a suffix of H. It is a simple inductive argument to show that Herq is indeed 
a probabilistic execution fragment of M. Observe that the definition of a suffix for ordinary 


executions is a special case of the definition we have just given. 


Example 4.2.8 (Suffixes) The probabilistic execution fragment Hz of Figure 4-9 is an ex- 
ample of a suffix. Specifically, Hy = H">(soas2), where H” is represented in Figure 4-8. The 
suffixing operation essentially extracts the subtree of H” that starts with sgasg and removes 
from each state the prefix soasg. | 


It is easy to check that the probability spaces Pyp, and Py), are in a one-to-one correspondence 
through the measurable function f : Q75q — Qyy, such that for each a € QAp,, f(a) = 97 a. 
The inverse of f is also measurable and associates apg with each execution a of Q7;),. Thus, 
directly from Proposition 4.2.11, we get the following proposition. 


Proposition 4.2.12 Let H be a probabilistic execution fragment of a probabilistic automaton 
M, and let q¢ be either a state of H, or a prefix of the start state of H. Then, for each subset 
E of QHg: 


Ll. FEC Frog iff (q° EF) € Fu. 
2. If E is an event, then Py|q~ E] = Py[Cq|Proql[E]. = 


4.2.7 Notation for Transitions 


In this section we extend the arrow notation for transitions that is used for ordinary automata. 
The extension that we present is meaningful for simple transitions only. 

An alternative representation for a simple transition (s,a,P) of a probabilistic automaton M 
is s —+ P. Thus, differently from the non-probabilistic case, a transition leads to a distribution 
over states. If P is a Dirac distribution, say D(s’), then we can represent the corresponding 
transition by s —> s’. Thus, the notation for ordinary automata becomes a special case of the 
notation for probabilistic automata. If (s,a,P) is a simple combined transition of M, then we 
represent the transition alternatively by s +c P, where the letter C stands for “combined”. 

The extension of weak transitions is more complicated. The expression s => P means 
that P is reached from s through a sequence of transitions of M7, some of which are internal. 
The main difference from the non-probabilistic case is that in the probabilistic framework the 
transitions involved form a tree rather than a linear chain. Formally, s => P, where a is either 
an external action or the empty sequence and P is a probability distribution over states, iff 
there is a probabilistic execution fragment H such that 


1. the start state of H is s; 
2. Pyl{aé | aé € Qy}] = 1, ie., the probability of termination in H is 1; 


3. for each ad € Qy, trace(a) = a; 
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Figure 4-10: A representation of a weak transition with action a. 
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Figure 4-11: A weak transition of a probabilistic automaton with cycles. 


4. P = Istate(6-strip(Py)), where é-strip(Py) is the probability space P’ such that Q/ = 
{a | aé € Oy}, and for each a € 0’, P’[a] = Py[Cas]; 


5. for each state q of H, either tr?! is the pair (Istate(q),DP(6)), or the transition that corre- 
sponds to tri? is a transition of M. 


A weak combined transition, s —c P, is defined as a weak transition by dropping Condition 5. 
Throughout the thesis we also the extend the function 6-strip to extended execution fragment; 
its action is to remove the symbol é at the end of each extended execution fragment. 


Example 4.2.9 (Weak transitions) Figure 4-10 represents a weak transition with action 
a that leads to state s; with probability 5/12 and to state s. with probability 7/12. The 
action 7 represents any internal action. From the formal definition of a weak transition, a tree 
that represents a weak transition may have an infinite branching structure, i.e., it may have 
transitions that lead to countably many states, and may have some infinite paths; however, the 
set of infinite paths has probability 0. 

Figure 4-11 represents a weak transition of a probabilistic automaton with cycles in its 
transition relation. Specifically, H represents the weak transition so => P, where P[so] = 1/8 
and P[s|] = 7/8. If we extend H indefinitely on its right, then we obtain a new probabilistic 
execution fragment that represents the weak transition so => D(s,). Observe that the new 
probabilistic execution fragment has an infinite path that occurs with probability 0. Further- 
more, observe that there is no other way to reach state s; with probability 1. | 


Remark 4.2.10 According to our definition, a weak transition can be obtained by concatenat- 


ing together infinitely many transitions of a probabilistic automaton. A reasonable objection 
to this definition is that sometimes scheduling infinitely many transitions is unfeasible. In the 
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timed framework this problem is even more important since it is feasible to assume that there 
is some limit to the number of transitions that can be scheduled in a finite time. Thus, a more 
reasonable and intuitive definition of a weak transition would require the probabilistic execution 
fragment H that represent a weak transition not to have any infinite path. All the results that 
we prove in this thesis are valid for the more general definition where H can have infinite paths 
as well as for the stricter definition where H does not have any infinite path. Therefore, we use 
the more general definition throughout. The reader is free to think of the simpler definition to 
get a better intuition of what happens. | 


An alternative way to represent a weak transition, which is used to prove the theorems of 
Chapter 8, is by means of a generator. If H represents a weak combined transition, then a 
generator can be seen as an object that chooses the combined transitions of M that lead to H 
(in Chapter 5 this object is also called an adversary). More precisely, a generator is a function 
O that associates a weak combined transition of M4 with each finite execution fragment of 
M. Before stating the formal properties that a generator satisfies, we give an example of the 
generator for the weak transition of Figure 4-10. 


Example 4.2.11 (Generators) Recall from Section 3.1.10 that U(a, y) denotes the probabil- 
ity space that assigns 2 and y probability 1/2 each. Then, the generator for the weak transition 
of Figure 4-10 is the function O where 


i 

3 T 

O(srst) = (sh,a,U(sh,8')) Olsrsasi) = (s.r, 

O(s) = (8,7,U(si,8h)) Olsrsh) =(sh,7,D(sh)) Olsrshyrsh) = (s4,a,U(s1, 82) 


and O(a) = (Istate(a),D(é)) for each a that is not considered above. The layout of the 
definition above reflects the shape of the probabilistic execution fragment of Figure 4-10. 
Thus, if we denote the probabilistic execution fragment of Figure 4-10 by H, O is the function 
that for each state q of H gives the combined transition of M that corresponds to tr?. Function 
O is also minimal in the sense that it returns a transition different from (lstate(q),D(6)) only 
from those states q that are relevant for the construction of H. We call active all the states of 
HT that enable some transition; we call reachable all the reachable states of H; we call terminal 
all the states g of H such that 6 € QF, | 


Let M be a probabilistic automaton and let s be a state of M. A generator for a weak 
(combined) transition s “124™) D of M is a function O that associates a (combined) transition 


of M with each finite execution fragment of M such that the following conditions are satisfied. 
1. If O(a) = (s’, P), then s’ = Istate(a). Call a active if P # D(6). 
2. If abs’ is active, then fstate(a) = s and (b, 8’) € Qera)- 


3. Call a reachable iff either a = s or a = a’bs’ and (b, 8’) € Qo(ar. Call a terminal iff a is 
reachable and Povaas')[6] > 0. Then, for each terminal a, the trace of a is a [ ext(M). 


4. For each reachable execution fragment a = sa1514252---a,5,, let 


Pe = II Po (sayy --a¢9;) (G41 $i41 DI], 
O<ick 
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Then, 
Q = {Istate(a) | terminal(a)}, 
and for each s’ € Q, 


P[s']= > PY Po(ay[6]- 


alistate(a)=s' ,terminal(a) 


Condition 1 says that the transition that O(a) returns is a legal transition of M from Istate(a); 
Condition 2 guarantees that the active execution fragments are exactly those that are relevant 
for the weak transition denoted by O; Condition 3 ensures that the weak transition represented 
by O has action a | ext(M); Condition 4 computes the probability space reached in the tran- 
sition represented by O, which must coincide with P. The term P° represents the probability 
of performing a if O resolves the nondeterminism in M. Observe that terminal execution frag- 
ments must be reachable with probability 1 if we want the structure computed in Condition 4 
to be a probability space. 


Proposition 4.2.13 There is a weak combined transition s => P of M iff there is a function 
O that satisfies the five conditions of the definition of a generator. 


Proof. Simple analysis of the definitions. | 


4.3. Parallel Composition 


In this section we extend to the probabilistic framework the parallel composition operator and 
the notion of a projection of ordinary automata. The parallel composition of simple probabilistic 
automata can be defined easily by enforcing synchronization on the common actions as in the 
non-probabilistic case; for general probabilistic automata, however, it is not clear how to give 
a synchronization rule. We discuss the problems involved at the end of the section. 


4.3.1 Parallel Composition of Simple Probabilistic Automata 


Two probabilistic automata M, and Mz are compatible iff 
int(M,) M acts( M2) = @ and acts(M,) A int( M2) = 0. 


The parallel composition of two compatible simple probabilistic automata M, and M2, denoted 
by M;||Mo, is the simple probabilistic automaton M such that 


1. states(M) = states(.M,) x states( M2). 
2. start(M) = start(M,) x start(M2). 
3. sig(M) = (eat(.My) U eat( M2), int(M,) U int(M2)). 


4. ((81, 82),a,P) € trans(M) iff P = P, ® Pz where 
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‘ (5135520) 
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12 
(577,520) a> (811,527) 
a WN 
1/2 a b 
(5).95 520) b> (S;2, 520) Zs (81,4529) ———> (575,520) 
1/2 
d 
a c 

(81,2 ,82,.) ———> (81,4, 52,1) > (576,527) 


Figure 4-12: A probabilistic execution fragment of Mq||Mo. 
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b 
12 
12 
S71 8 51,5 
b 
12 1/2 
12 12 
Sho b> 512 a> S14 on S16 


Figure 4-13: The projection onto M4, of the probabilistic execution fragment of Figure 4-10. 


(a) if a € acts(M,) then (s1,a,P1) € trans(M,), else P; = D(s1), and 
(b) if a € acts( M2) then (s2,a,P2) € trans( M2), else Pz = D(s2). 


Similar to the non-probabilistic case, two simple probabilistic automata synchronize on their 
common actions and evolve independently on the others. Whenever a synchronization occurs, 
the state that is reached is obtained by choosing a state independently for each of the proba- 
bilistic automata involved. 


4.3.2 Projection of Probabilistic Executions 
The Structure of the Problem 


Let M = M;||Mg, and let H bea probabilistic execution fragment of M. We want to determine 
the view that My, has of A, or, in other words, what probabilistic execution AM, performs in 
order for M,||Mz to produce H. To understand the complexity of the problem, consider the 
probabilistic execution fragment of Figure 4-12, and consider its projection onto Mj, represented 
in Figure 4-13. Actions a,6 and c are actions of M1, while action dis an action of M2. Thus, 
there is no communication between My, and Mz. Denote the probabilistic execution fragment 
of Figure 4-12 by H, and denote the probabilistic execution fragment of Figure 4-13 by Hy. 
The projections of the states are ordinary projections of pairs onto their first component. The 
transitions, however, are harder to understand. We analyze them one by one. 


S,,. The transition leaving s19 is obtained directly from the transition leaving (s1,9, 42,9) in 
HT by projecting onto M, the target states. 


8,2 The transition leaving 51,2 is obtained by combining the transitions leaving states (51,2, 52,0) 
and (51.2, 82,1), each one with probability 1/2. The two transitions leaving (51,2, 52,9) and 
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(51,2, 52,1) have the same projection onto M,, and thus the transition leaving s12 in Hy 
is 812 —, 81,4. From the point of view of M,, there is just a transition 52 —, $145 
nothing is visible about the behavior of Mo. 


To give a better idea of what we mean by “visible”, suppose that M4, is a student who 
has to write a report and suppose that the report can be written using a pen (action 
c) or using a pencil (action b). Suppose that the teacher may be able to get a pencil 
eraser (action d) and possibly erase the report written by the student once it is ready for 
grading. Then the scheduler is an arbiter who gives the student a pen if the teacher gets 
an eraser. If the student starts in state 512, then from the point of view of the student 
the material for the report is prepared (action a), and then the arbiter gives the student 
a pen with probability 1/2 and a pencil with probability 1/2; nothing is known about the 
time the the arbiter made the choice and the reason for which the choice was made. We 
can also think of the student as being alone in a room and the arbiter as being a person 
who brings to the student either a pen or a pencil once the material for the report is 
ready. 


The detailed computation of the transition leaving from s1,2 in H, works as follows: we 
start from state (51,2, 52,9), which is the first state reached in H where M, is in 512, and 
we analyze its outgoing edges. We include directly all the edges labeled with actions of 
My, in the transition leaving s1,2; for the other edges, we move to the states that they 
lead to, in our case (51,2, 52,1), and we repeat the same procedure keeping in mind that 
the probability of the new edges must be multiplied by the probability of reaching the 
state under consideration. Thus, the edge labeled with a that leaves (s1,2, 52,9) is given 
probability 1/2 since its probability is 1/2, and the edge that leaves (51,2, 52,1) is given 
probability 1/2 since the probability of reaching (51,2, 52,1) from (s1,2, 2,9) is 1/2. 


s,,, For the transition leaving s;,4, we observe that in H there are two states, namely (51,4, 52,0) 
and (1,4, 42,1), that can be reached separately and whose first component is 51,4. Each 
one of the two states is reached in H with probability 1/4. The difference between the 
case for state 5; and this case is that in the case for 51 state (51,2, 2,9) occurs before 
(51,2, 52,1), While in this case there is no relationship between the occurrences of (1,4, 52,0), 
and (514,52). The transition leaving s;,4 depends on the state of Mz which, conditional 
on M, being in s1,4, is 1/2 for sy and 1/2 for s2,. Thus, from the point of view of My, 
since the state of M2 is unknown, there is a transition from s;,4 that with probability 1/2 
leads to the occurrence of action b and with probability 1/2 leads to the occurrence of 
action c. Essentially we have normalized to 1 the probabilities of states (51,4, $29) and 
(51,4, 52,1) before considering their effect on M4. 


s,,, The transition leaving s;,, shows why we need the symbol 6 in the transitions of a proba- 
bilistic automaton. From state (511, 52,9) there is a transition where action b occurs with 
probability 1/2 and action 7 occurs with probability 1/2. After 7 is performed, nothing 
is scheduled. Thus, from the point of view of M,, nothing is scheduled from 5, with 
probability 1/2; the transition of M2 is not visible by Mj. 
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Action Restricted Transitions 


The formal definition of a projection relies on a new operation on transitions, called action 
restriction, which is used also in several other parts of the thesis. The action restriction op- 
eration allows us to consider only those edges of a transition that are labeled with actions 
from a designated set V. For example, V could be the set of actions of a specific probabilistic 
automaton. 

Formally, let M be a probabilistic automaton, V be a set of actions of M, and tr = (s,P) 
be a transition of @. The transition tr restricted to actions from V, denoted by tr f V, is 
the pair (s,P’) where P’ is obtained from P by considering only the edges labeled with actions 
from V and by normalizing their probability to 1, ie., 


6 Oe {(a,s')EQ|aEeV} if PIV] >0 
~ | {6} otherwise 


e if P[V] > 0, then for each (a, s’) € 0’, P'[(a, s’)] = Pl(a, s‘)]/P[V]. 


Two properties of action restriction concern commutativity with transition prefixing, and dis- 
tributivity with respect to combination of transitions. These properties are used in the proofs 
of other important results of this thesis. The reader may skip the formal statements for the 
moment and refer back to them when they are used. 


Proposition 4.3.1 For each g and tr such that one of the expressions below is defined, 
q° (tr TV) = (> tr) TV. 
Proof. Simple manipulation of the definitions. | 


Proposition 4.3.2 Let {t;}ic7 be a collection of transitions leaving from a given state s, and 
let {pitier be a collection of real numbers between 0 and 1 such that Vjeppi < 1. Let V be a 
set of actions. Then 


Ltr. DiP wr [V : 
(opine) FV == =, oe tin tv). 


where we use the convention that 0/0 = 0. 


Proof. Let 
(s,P) = > Upitri. (4.15) 
(s,P') = (CS ipitri) PV, (4.16) 
" A piPy [VY] r 
(PN Pen TY) (417) 


We need to show that P’ and P” are the same probability space. 


64 


If P[V] = 0, then both P’ and P” are D(6) and we are done. Otherwise, observe that 
neither 9’ nor 2” contain 6. Consider any pair (a, s’). Then, 


(a, 8’) EO 
iff (a,s')€ QandaeV from (4.16) and (4.15) 
iff S;(a,s') € Q;,p; > 0, anda eV from (4.15) 
iff S;(a,s’) € Qe, pv and p; > 0 from the definition of tr; | V 
iff (a,s')E QQ” from (4.17). 


Consider now a pair (a, s’) of 2’. From the definition of action restriction and (4.16), 


P(a, 8] = Pl(a,s)]/ PIV]. (4.18) 
From the definition of P (Equation (4.15)), the right side of Equation 4.18 can be rewritten 


into 


dX Spray [(a.8’)], (4.19) 


where >>, p;Pi,,[V] is an alternative expression of P[V] that follows directly from (4.16). By 
multiplying and dividing each i** summand of Expression 4.19 by P;,,[V], we obtain 


DiPer, [Vv] 
F wi PiP WV] 
Since Pi,,[(a,8’)]/Pr,[V] = Pir, pv[(a, 3’)], from the definition of P” (Equation (4.17)), Expres- 


sion 4.20 can be rewritten into P”[(a,s’)]. Thus, P’[(a,s’)] = P”[(a,s’)]. This is enough to 
show that P’ = P”. a 


(Per: [(, 8)]/ Pr [V])- (4.20) 


Definition of Projection 


We give first the formal definition of a projection, and then we illustrate its critical parts by 
analyzing the example of Figures 4-12 and 4-13. It is very important to understand Expres- 
sions (4.21) and (4.22) since similar expressions will be used in several other parts of the thesis 
without any further explanation except for formal proofs. 

Let M = M,||Mo, and let H be a probabilistic execution fragment of M. 

Let tr = (q,P) be an action restricted transition of H such that only actions of M;, i = 1,2, 
appear in tr. Define the projection operator on the elements of 2 as follows: (a,q')[M; = 
(a,q'[M;), and 6/M; = 6. Recall from Section 3.1.5 that the projection can be extended 
to discrete probability spaces. The projection of tr onto M;, denoted by tr[|M;, is the pair 
(q[Mi, P[M;). 

The projection of H onto M;, denoted by H[M;, is the fully probabilistic automaton H’ 
such that 


1. states( H1') = {q[M; | ¢ € states( H)}; 
2. start( H') = {q[M; | ¢ € start(H)}; 


3. sig( H’) = sig( Mj); 
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4. for each state g of H’, let q|H be the set of states of H that projected onto M; give gq, 
and let min(q|H) be the set of minimal states of g|H under prefix ordering. For each 
q¢ € (q|H), let 


aes PulCal 


Py — oo (4.21) 
qd digemin(q]H) Pr[Cq"] 
The transition enabled from q¢ in H’ is 
trl’ 2 S> pt] PH (acts(M,)\(tr } acts(M;)) [My (4.22) 


Each summand of Expression 4.22 corresponds to the analysis of one of the states of H that can 
influence the transition enabled from gq in H’. The subexpression (int - acts(.M;))[M; selects 
the part of the transition leaving from q’ where M; is active, and projects onto M; the target 


states of the selected part; the subexpression pil" PH [acts( Mz) expresses the probability with 


which gq’ influences the transition enabled from q. PH [acts(M;)] is the probability that tr does 


something visible by M;, and pil is the probability of being in gq’ conditional on M; being in q. 


Its value is given by Expression 4.21 and can be understood as follows. The state q’ is either a 
minimal state of q|H or is reached from a minimal state through a sequence of edges with actions 
not in acts(M;). The probability of being in q’, conditional on M; being in q, is the normalized 
probability of being in the minimal state of g|H that precedes q’ multiplied by the probability 
of reaching q' from that minimal state. We encourage the reader to apply Expression (4.22) to 
the states 51,9, 51,1, $1,2, and 81,4 of Figure 4-13 to familiarize with the definition. As examples, 
observe that min((s1,0bs1,2)| 4) = {(s1,0, $2,0)b(s1,2, $2,0)} and that min((s1,9bs1,2451,4)|H) = 
{(S1,0; $2,0)b( 51,2, $2,0)@( 81,4, $2,0), (81,0, $2,0)0($1,2, $2,0)d( 1,2, 2,1) (81,4, 82,1) }. 

If we analyze the state s13 of Figure 4-13 and we use Expression 4.22 to compute the 
transition leaving s;,3, then we discover that the sum of the probabilities involved is not 1. 
This is because there is a part of the transition leaving (51,3, 2,9) where no action of My, ever 
occurs. From the point of view of M, nothing is scheduled; this is the reason of our choice of 
deadlock by default in the definition of the combination of transitions (cf. Section 4.2.2). 

We now move to Proposition 4.3.4, which is the equivalent of Proposition 3.2.1 for the 
probabilistic framework. Specifically, we show that the projection of a probabilistic execution 
fragment H of M,||M2 onto one of its components M; is a probabilistic execution fragment 
of M;. Proposition 3.2.1 is important because it shows that every computation of a parallel 
composition is the result of some computation of each of the components. One of the reasons 
for our use of randomized schedulers in the model is to make sure that Proposition 3.2.1 is 
valid. Before proving this result, we show that its converse does not hold, i.e., that there are 
structures that look like a probabilistic execution, that projected onto each component give a 
probabilistic execution of a component, but that are not probabilistic executions themselves. 


Example 4.3.1 (Failure of the converse of Proposition 4.3.4) Consider the probabilis- 
tic automata of Figure 4-14.a, and consider a potential probabilistic execution of the composi- 
tion as represented in Figure 4-14.b. Denote the two probabilistic automata of Figure 4-14.a by 
My, and Mo, and denote the structure of Figure 4-14.b by H. The projections of H onto M, and 
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a) Two compatible simple probabilistic automata. 


da 
: (59, S;) ———> (5,, 5) 
12 


SoS 
( 0> 0) In 


(59, 52) qo (52, 52) 


b) A potential probabilistic execution of the composition. 


Figure 4-14: A counterexample to the converse of the projection proposition. 


My give a probabilistic execution of M, and Mo, respectively. The diagrams of Figure 4-14.a 
can be viewed as the projections of H as well. However, A is not a probabilistic execution of 
M,\||Mo since in no place of M, it is possible to have a Dirac transition to 51 or 52. a 


The rest of this section is dedicated to the proof of the proposition that corresponds to Propo- 
sition 3.2.1 and to the proof of an additional result (Proposition 4.3.5) that gives a meaning to 
the denominator of Expression (4.21). We first state two preliminary properties of projection 
of transitions (Proposition 4.3.3). 


Proposition 4.3.3 Let M = M,||M2. Then, for i = 1,2, 
1. (02; pitrs) [Mi = Xj pi(tr;[ Mi). 
2 (q~ tr)[M; = (q[M,) * tr[M;. 
Proof. Simple manipulation of the definitions. | 


Proposition 4.3.4 Let M = M,||Mz, and let H be a probabilistic execution fragment of M. 
Then H|M, € prexec(M,) and H| Mz © prexec( M2). 


Proof. We show that H|M, © prexec(M,); the other statement follows from a symmetric 
argument. Let H, denote H/M,. From Proposition 3.2.1, the states of H, are execution 
fragments of My. 

Consider now a state q of H,. We need to show that there is a combined transition tr of 
M, that corresponds to ae i.e., such that ine = qtr. From Propositions 4.2.1 and 4.2.3, 
it is sufficient to show that for each state q’ of q|H, there is a combined transition tr(q’) of My 
such that 


(ir t acts(M1))[M1 = ¢7 tr(q’). (4.23) 
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Then, the transition tr would be 


tr= 32 pil" P¥lacts(M;)]tr(q). (4.24) 
q’€q|H 
Proposition 4.2.1 is used to show that tr is a combined transition of M,; Proposition 4.2.3 is 


used to show that q7 tr = tft, Since H is a probabilistic execution fragment of M, for each 
state q’ of g|H there exists a combined transition tr’(q’) of M such that 


tr =~ tr'(q). (4.25) 


From the definition of a combined transition, there is a collection of transitions {tr’(q’,7)}ier 
of M, and a collection of probabilities {p;};e7, such that 


= dpitr'(d, i). (4.26) 


Note that each transition tr’(q’,7) is a simple transition. From the definition of action restriction 
and (4.26), there is a subset J of I, and a collection of non-zero probabilities {pj} jes, such that 


tr'(q') | acts(M;) =n tr'(q',j). (4.27) 


If we apply transition prefix with q’ to both sides of Equation 4.27, we use commutativity 
of action restriction with respect to transition prefixing (Proposition 4.3.1) and (4.25) on the 
left expression, and we use distributivity of transition prefixing with respect to combination of 
transitions (Proposition 4.2.3) on the right expression, then we obtain 


tr | acts(M,) =n (a'" qd, (q'.3))- (4.28) 


By projecting buth sides of (4.28) onto Mj, and using distributivity of projection with respect to 
combination of transitions (Proposition 4.3.3) and commutativity of projection and transition 
prefixing (Proposition 4.3.3) on the right expression, we obtain 


(tr! } acts(M,)) =e vi (a7 q'.3)[Mi)) (4.29) 


From the distributivity of transition prefixing with respect to combination of transitions (Propo- 
sition 4.2.3), Equation 4.29 becomes 


(iri | acts(M,)){[My = q7 dpi (ir'(7, 5) Mi). (4.30) 


From standard properties of the projection of product probability distributions (cf. Sec- 
tion 3.1.6) and the definition of parallel composition, each tr’(q’,7)[Mi is a transition of M,. 
Thus, 3°; pjtr'(q’, 7) [M1 is the combined transition of M, that satisfies Equation 4.23. 
Finally, we need to show that each state q of Hy is reachable. This is shown by induction 
on the length of g, where the base case is the start state of H,. The start state of Hy is 
trivially reachable. Consider a state gas of H,. By induction, q is reachable. Let q’ be a 
minimal state of (qas)|H. Then, q' = q"a(s, sz), where q” is a state of g]H and s2 is a state 
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of Mz. Moreover, (a,q') € Q,,#, and thus, (a, gas) € QU in pacts(My))[My: >itice no edges with 
gt gt 


probability 0 are allowed in a probabilistic automaton, the term pil" PH [acts( M;)] is not 0, 
and thus (a, qas) € OF, This means that gas is reachable. a 


We conclude this section with another property of projections that gives a meaning to the 
denominator of Expression (4.21). Specifically, the proposition below allows us to compute the 
probability of a finitely satisfiable event of the projection of a probabilistic execution fragment 
HT by computing the probability of a finitely satisfiable event of H. Observe that the right 
expression of (4.31) is indeed the denominator of (4.21). 


Proposition 4.3.5 Let M = M,||Mz, and let H be a probabilistic execution fragment of M. 
Let H; be H|M;, i = 1,2. Let q be a state of H;. Then, 


PalCg= 2 PalCe. (4.31) 
q'Emin(q|H) 
Proof. The proof is by induction on the length of g, where the base case is for the start state 
of H;. If q is the start state of H;, then the start state of H is the only minimal state of q|H. 
Both the cones denoted by the two states have probability 1. 
Consider now the case for gas. From the definition of the probability of a cone, 


Pr, [Coas] = Pr, [C,]Pi" [(a, qas)]. (4.32) 


By using Expression 4.22 and the definitions of action restriction and projection, the term 
P#4\[(a,qas)] can be rewritten into 


S- pil" P [acts( M;)] S- PH (a, q)\/ Pj [acts(M;)] | . (4.33) 
Veq|H q"€(qas)|H|(a,a" EQ 


which becomes 


ype > PH((a,q"\]] . (4.34) 


gVe€q|H q"€(qas)|A|(a,q" EQ", 


after simplifying the term Pi lacts(M;)]. The case when Pi lacts( M;)] = 0 is not a problem 
since the innermost sum of Expression 4.33 would be empty. By expanding pil in Expres- 
sion 4.34 with its definition (Equation 4.21), applying induction to Py,[C,] in Expression 4.32, 


and simplifying algebraically, Equation 4.32 can be rewritten into 


Pr, [Cqas] = > > Py[Cy\ Py [(a, q'))- (4.35) 
Eq q!E(qas)|H|(a,q"EQH 


| 


Indeed, the denominator of the expansion of Py! coincides with the expansion of Py, [C4]. 


From the definition of the probability of a cone, the terms Pr[Cy\ Pi [(a, q’)| that appear 
in Equation 4.35 can be rewritten into Py[C,”]. 
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Consider now one of the states q” of the right side of Equation 4.35. Then q"”[M; = gas, and 
there exists a state q' of q|H such that (a,q”) € Q 4. This means that q” can be expressed as 
q'as' for some state s’ of M. Since q’[M; = q, then q” is a minimal state of (gas)| H. Conversely, 
let q” be a minimal state of (qas)|H. Then q” can be expressed as q’as’ for some state q' of H 
and some state s’ of M (otherwise g” would not be minimal). Moreover, q’ is a state of g|H 
and (a,q") € QF, Thus, q” is considered in Equation 4.35. Finally, each minimal state q” of 
(qas)|H is considered at most once in Equation 4.35, since there is at most one state q’ in H 
such that (a, q") € QF, Thus, Equation 4.35 can be rewritten into 


Pr, [Cgas] = Se. PalCa (4.36) 
q/Emin((qas)|H) 


which is what we needed to show. | 


4.3.3. Parallel Composition for General Probabilistic Automata 


In this section we give an idea of the problems that arise in defining parallel composition for 
general probabilistic automata. The discussion is rather informal: we want to give just an idea 
of why our intuition does not work in this case. 

The main problem that needs to be addressed is to choose when two transitions should 
synchronize and how the synchronization would occur. We analyze the problem through some 
toy examples. Consider two probabilistic automata M1, Mz with no internal actions and such 
that ert(M,) = {a,b,c,d} and ext( M2) = {a,b,c,e}. Let (51,52) bea reachable state of My || Mo, 
and consider the following cases. 


1. Suppose that from state s, of M, there is a transition tr, giving actions a,b probability 
1/2 to occur, and suppose that from state s2 of M3 there is a transition tr2 giving actions 
a,b probability 1/2 to occur. 


If we choose not to synchronize tr; and trg, then the only transitions that can be syn- 
chronized are the simple transitions, leading to a trivial parallel composition operator 
that does not handle any kind of transition with probabilistic choices over actions. The 
transitions try and trg cannot be scheduled even independently, since otherwise the CSP 
synchronization style would be violated. 


If we choose to synchronize tr; and tra, then both My, and M2 choose an action between 
a and b. If the actions coincide, then there is a synchronization, otherwise we have two 
possible choices in our definition: either the system deadlocks, or the random draws are 
repeated. The first approach coincides with viewing each probabilistic automaton as de- 
ciding its next action probabilistically independently of the other interacting automaton; 
the second approach is the one outlined in [GSST90], where essentially deadlock is not 
allowed, and assumes some dependence between the involved probabilistic automata. 


For the rest of the discussion we assume that the transitions try; and trg do synchronize; 
however, we leave unspecified the way in which tr, and tra synchronize. 
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2. Suppose that from state s; of M4, there is a transition tr, giving actions a,b probability 
1/2 to occur, and suppose that from state sz of M3 there is a transition tr2 giving actions 
a,c probability 1/2 to occur. 


Note that actions a,b and ¢ are all in common between M, and Mp). If we choose not 
to synchronize tr; and tr2, then only transitions involving the same sets of actions can 
synchronize. However, we have the same problem outlined in Case 1, where neither try, 
nor trg can be scheduled independently. 


If we choose to synchronize tr; and tr2, then, since a is the only action that is in common 
between tr, and tr2, the only action that can occur is a. Its probability is either 1 or 1/4 
depending on how the synchronization in Case | is resolved. However, in both cases the 
only action that appears in the sample space of the composite transition is a. 


For the rest of the discussion we assume that the transitions try; and tra do synchronize. 
Once again, we leave unspecified the way in which tr; and tr synchronize. 


3. Suppose that from state s, of M, there is a transition tr, giving actions a,b, d probability 
1/3 to occur, and suppose that from state s2 of M3 there is a transition tr2 giving actions 
a,b,e probability 1/3 to occur. 


In this case each transition has some actions that are in common between M, and Mo, 
and some actions that are not in common. 


If we choose not to synchronize tr, and tra, then, beside the fact that tr, and tre could not 
be scheduled independently, the parallel composition operator would not be associative. 
Consider two new probabilistic automata Mj, Mj with the same actions as M, and Mg, 
respectively. Suppose that from state s{ of Mj there is a transition tr‘, giving actions a, b 
probability 1/2 to occur, and suppose that from state s4 of M3 there is a transition tr 
giving actions a,b probability 1/2 to occur. 


If we consider (My||My)||(M2||.W3), then in state ((s/, 51), (s2, 84)) tr1 would synchronize 
with tr{ leading to a transition that involves actions a and 6 only, tr2 would synchronize 
with tr, leading to a transition that involves actions a and 6 only, and the two new 
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transitions would synchronize because of Case 1, leading to a transition that involves 
actions a and 6. If we consider (M{||(Mi||Mz2))|| 443, then in state ((s4,(s1,52)), 9) tr1 
and trg would not synchronize, and thus associativity is broken. 


If we choose to synchronize tr, and tr2, then problems arise due to the presence of actions 
that are not in common between M, and Mo. In particular we do not know what to do if 
M, draws action d and Mz draws action e, or if M, draws action d and M> draws action 
a. Since we do not want to assume anything about the respective probabilistic behaviors 
of M, and Mg, at least the first case is an evident case of nondeterminism. 


However, even by dealing with the first case above by means of nondeterminism, only 
one of actions d,e can be performed. Suppose that d is chosen, and thus Mj, performs a 
transition while M, does not. What happens to M2? Is action e supposed to be chosen 
already after d is performed? Otherwise, what is the probability for e to occur? At this 
point we do not see any choice that would coincide with any reasonable intuition about 
the involved systems. 


In the second case we are sure that action a cannot occur. Does this mean that action d 
occurs for sure? Or does this mean that a deadlock can occur? With what probabilities? 
Once again, intuition does not help in this case. 


The main problem, which is evident especially from Case 3, is that we do not know who is in 
control of a system, and thus, whenever there is a conflict that is not solved by nondeterminism 
alone, we do not know what probability distribution to use to resolve the conflict. However, 
if we decorate probabilistic automata with some additional structure that clarifies who is in 
control of what actions [LT87], then parallel composition can be extended safely to some forms 
of general probabilistic automata, where the external actions are partitioned into input and 
output actions, the transitions that contain some input action are simple transitions, and input 
actions are enabled from every state (cf. Section 13.2.2). An observation along this line appears 


in [WSS94]. 


4.4 Other Useful Operators 


There are two other operators on probabilistic automata that should be mentioned, since they 
are used in general on ordinary automata. In this section we provide a short description of 
those operators. Since the relative theory is simple, this is the only point where we mention 
these operators during the development of the probabilistic model. 


4.4.1 Action Renaming 


Let p be a one-to-one function whose domain is acts(M). Define Rename,(M) to be the 
probabilistic automaton M’ such that 


1. states( M’) = states(M). 
2. start(M’) = start(M). 


3. sig(M’) = (p(ext(M)), plint(.M1))). 
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4. (s,P) € trans(M’) iff there exists a transition (s,P’) of M such that P = p’(P’), where 
p’((a, 8')) = (p(a), s’) for each (a, s’) € 0’, and p’(é) = 6. 


Thus, the effect of Rename, is to change the action names of M. The restriction on p to be 
one-to-one can be relaxed as long as internal and external actions are not mixed, i.e., there is 
no pair of actions a,6 where a is an external action, 6 is an internal action, and p(a) = p(6). 


4.4.2 Action Hiding 


Let M be a probabilistic automaton, and let J be a set of actions. Then Hide;(M) is defined 
to be a probabilistic automaton M’ that is the same as M, except that 


sig( M") = (ext(M) —T, int(M) UT). 


That is, the actions in the set J are hidden from the external environment. 


4.5 Discussion 


The generative model of probabilistic processes of van Glabbeek et al. [GSST90] is a special 
case of a fully probabilistic automaton; simple probabilistic automata are partially captured 
by the reactive model of [GSST90] in the sense that the reactive model assumes some form 
of nondeterminism between different actions. However, the reactive model does not allow 
nondeterministic choices between transitions involving the same action. By restricting simple 
probabilistic automata to have finitely many states, we obtain objects with a structure similar to 
that of the Concurrent Labeled Markov Chains of [Han91]; however, in our model we do not need 
to distinguish between nondeterministic and probabilistic states. In our model nondeterminism 
is obtained by means of the structure of the transition relation. This allows us to retain most 
of the traditional notation that is used for automata. 

Our parallel composition operator is defined only for simple probabilistic automata, and thus 
a natural objection is that after all we are dealing just with the reactive model. Furthermore, 
the reactive model is the least general according to [GSST90]. Although we recognize that our 
simple probabilistic automata constitute a restricted model and that it would be desirable to 
extend the parallel composition operator to general probabilistic automata, we do not think that 
it is possible to use the classification of [GSST90] to judge the expressivity of simple probabilistic 
automata. The classification of [GSST90] is based on a synchronous parallel composition, while 
our parallel composition is based on a conservative extension of the parallel composition of CSP 
[Hoa85]. Furthermore, in the classification of [GSST90] a model is more general if it contains 
less nondeterminism, while in our model nondeterminism is one of the key features. 
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Chapter 5 


Direct Verication Stating a 
Property 


This chapter presents a method to study the properties that a probabilistic automaton satisfies. 
We describe how an informally stated property can be made rigorous, and we show how simple 
statements can be combined together to give more complex statements. In Chapter 6 we develop 
techniques to prove from scratch that a probabilistic automaton satisfies a given property. 

Part of this chapter is based on discussion with Isaac Saias who provided us with the 
motivations for the definition of progress statements (Section 5.5) and for the statement of the 
concatenation theorem (‘Theorem 5.5.2). 


5.1 The Method of Analysis 


If we read through the papers on randomized algorithms and we look at the statements of 
correctness, we see claims like 


“Whenever the algorithm X starts in a condition Y, no matter what the adversary 
does, the algorithm X achieves the goal Z with probability at least p.” 


For convenience, denote the statement above by S$. A possible concrete instantiation of $ is 
the following: 


“Consider a distributed system X, composed of n processors, that provides services 
under request and suppose that some request R comes. Then, independently of the 
relative order in which the n processors complete their operations (no matter what 
the adversary does), a response to R is given eventually (the goal Z) with probability 
at least 2/3. 


Let us try to understand the meaning of the statement S. First of all, in S there is an entity, 
called adversary, that affects the performance of algorithm X. The adversary is seen as a 
malicious entity that degrades the performance of X as much as possible. 

If X is a distributed algorithm that runs on n separate processes, then the adversary is the 
entity that chooses what process performs the next transition, and possibly what the external 
environment does. To account for all the possible ways to schedule processes, the adversary 
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Figure 5-1: A toy resource allocation protocol. 
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bases its choices on a complete knowledge of the state of a system, including its past history. If 
the algorithm is represented as a probabilistic automaton, then an adversary is the object that 
resolves the nondeterminism. In other words, an adversary is a scheduler seen as a malicious 
entity. 

However, not all the schedulers guarantee in general that some specific property is satisfied. 
For example, an adversary is usually required to be fair to all the processes of a system in 
order to guarantee progress. In other cases, an adversary is not allowed to base its choices on a 
complete knowledge of the history of a system: the correctness of an algorithm may rely on the 
adversary not to use the results of previous random draws in choosing the next process to be 
scheduled. Thus, in the statement $ there is usually an implicit assumption that an adversary 
has some limitations. 


Example 5.1.1 (A toy resource allocation protocol) Figure 5-1 illustrates a toy scenario 
where correctness is guaranteed only for adversaries that do not know the outcome of the random 
draws of the processes. Two processes M, and M2 compete for two resources Ry and Rg. Each 
process continuously runs through the following cycle: 


1. flip a coin to choose a resource; 
2. if the chosen resource is free, then get it; 
3. if you hold the resource, then return it. 


That is, each process continuously tries to get a randomly chosen resource and then returns it, 
possibly after using the resource. Of course this is a stupid protocol, but it highlights several 
aspects of randomized distributed algorithms. Suppose every adversary to be fair, meaning 
that both processes perform infinitely many transitions. A malicious adversary can create a 
situation where M, never succeeds in obtaining a resource with an arbitrarily high probability. 
The adversary works as follows. Fix an arbitrary probability p such that 0 < p < 1, and consider 
a collection of probabilities {p;};en such that [];p; = p. We know that such a collection 
of probabilities exists. Then the adversary works in rounds, where at round 7 the following 
happens: 


a. My, is scheduled until it flips its coin; 


b. Mz is scheduled for sufficiently many times so that it gets the resource chosen by M, 
with probability at least p; (finitely many times are sufficient). As soon as M2 gets the 
resource chosen by My the control goes to c; 
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c. My, is scheduled to check its resource and fails to get it. 


In this case My fails to obtain a resource with probability at least p. On the other hand, if 
an adversary is not allowed to base its choices on the outcome of the coin flips, or better, 
if an adversary chooses the next process that performs a transition based only on the order 
in which processes were scheduled in the past, then each process eventually gets a resource 
with probability 1 (this fact is proved in Section 6.6). Such an adversary is called an oblivious 
adversary or an off-line scheduler. | 


Let us move back to the problem of understanding the statement 5. Consider a valid adversary 
A, i.e., an adversary that satisfies the limitations that are implicitly assumed for S. Let M 
be a probabilistic automaton that describes algorithm X, and consider an arbitrary starting 
point g for M, i.e., q is a finite execution fragment of M that describes a partial evolution of 
M. If we let A resolve the nondeterminism in M starting from the knowledge that ¢g occurred, 
then we obtain a probabilistic execution fragment of M, which we denote by prexec( M,.A,q). 
According to S, if q satisfies condition Y, then prerec( M,A,q) should satisfy property 7 with 
probability at least p. However, Z is a property of WM, and not a property of prexec( M,.A,q). 
Thus, we need a way to associate with prerec(M,.A,q) the event that expresses 7. The object 
that does this operation is called an event schema. At this point it is possible to formalize $ 
by stating the following: 


“For each valid adversary A and each valid starting condition q, the probability of 
the event associated with prexec( M,A,q) is at least p.” 


This is an example of what we call a probabilistic statement. 
A probabilistic statement that plays an important role in our analysis is denoted by 


U Ades U', (5.1) 


where U and U’ are sets of states, p is a probability, and Advs is a set of adversaries. We call 
such a statement a progress statement. Its meaning is that if a protocol starts from a state of 
U, then, no matter what adversary of Advs is used to resolve the nondeterminism, some state of 
U’ is reached with probability at least p. A progress statement is a probabilistic generalization 
of the leads-to operator of UNITY [CM88]. 


Example 5.1.2 It is possible to show (cf. Section 6.6) that the toy resource allocation protocol 
satisfies 72 Tate Mj, where R is the set of reachable states of M1||M2, M, is the set of states 


of M,||M2z where M, holds a resource, and Advs is the set of fair oblivious and adversaries for 
M,||Mo, ie., the set of adversaries that are fair to each process and that do not base their 
choices on the outcomes of the coin flips (cf. Example 5.6.2 for a formal definition of a fair 
oblivious adversary). a 


Progress statements are important because, under some general conditions, they can be com- 
bined together to obtain more complex progress statements, thus allowing the decomposition 
of a complex problem into simpler problems. 
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Example 5.1.3 Suppose that in some system M whenever a request is pending (M is in a 
state of some set P, a token is given eventually with probability at least 1/2 (reaching a state 
of some set 7), and suppose that whenever a token is given a response is given eventually with 
probability at least 1/3 (reaching a state of some set G). That is, 


P —advs T and T —YJAdus G. (5.2) 
1/2 1/3 


Then, it is reasonable to conclude that whenever a request is pending a response is given 
eventually with probability at least 1/6, i-e., 


P Tees G. (5.3) 
This result is a consequence of the concatenation theorem (cf. Theorem 5.5.2). a 


Example 5.1.4 Consider the toy resource allocation protocol again. We know from Exam- 
ple 5.1.2 that 


R Tate My. (5.4) 


It is also possible to show that 
R => R Unless My, (5.5) 


where R > R Unless M, is a UNITY [CM88] expression stating that whenever a system is in a 
state of 2 the system remains in a state of R unless a state of My, is reached. This means that 
(5.4) is applicable from any point in the evolution of the toy resource allocation protocol, and 
this fact, together with the condition that every adversary is fair, is succicient to guarrantee 
that 


R yy Adus My (5.6) 


(cf. Proposition 5.5.6). The reader familiar with UNITY may note that the combination of 
(5.4) and (5.5) is a probabilistic generalization of the ensures operator of Chandy and Misra 
[CM88]. = 


To see more significative applications of progress statements the reader is referred to Chapter 6, 
where we prove the correctness of the randomized Dining Philosophers algorithm of Lehmann 
and Rabin [LR81], and we prove the correctness of the randomized algorithm of Ben-Or for 
agreement in asynchronous networks in the presence of stopping faults [BO83]. Instead, the final 
part of this chapter concentrates on standard methods to specify event schemas and adversary 
schemas, and on the relationship between deterministic and general (randomized) adversaries. 
The main lesson that we learn is that for a large class of probabilistic statements it is possible 
to prove their validity by considering only deterministic adversaries, i.e., adversaries that do 
not use randomization in their choices. The reader who is reading only the first section of each 
chapter should move to Chapter 6 at this point and skip the rest of this section. 

We said already that an event schema is a rule to associate an event with each probabilistic 
execution fragment. More formally, an event schema is a function that given a probabilistic 
execution fragment H returns an event of 77. However, we have not given any method to 
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specify an event schema. Our definition of an event schema is very general since it allows for 
any kind of rule to be used in determining the event associated with a probabilistic execution 
fragment. On the other hand, there is a specific rule which is used in most of the existing 
literature on randomized algorithms. Namely, given a probabilistic automaton M, a set O of 
execution fragments of M is fixed, and then, given a probabilistic execution fragment H of M, 
the event associated with H is ON Qy. We call such an event schema an erecution-based event 
schema. Since the start state of a probabilistic execution fragment contains part of the history of 
M, and since in general we are interested in what happens only after the probabilistic execution 
fragment starts, we refine the definition of an execution-based event schema by associating a 
probabilistic execution fragment H with the event ON (Qyegé?), where qé! is the start state of 
HT. In this way a progress statement can be stated in terms of execution-based event schemas, 
where © is the set of execution fragments of M that contain at least one occurrence of a state 
from U'. 

To specify an adversary schema there are two main restrictions that are usually imposed. 
One possibility is to restrict the kind of choices that an adversary can make, and the other 
possibility is to restrict the on-line information that an adversary can use in making its choices. 
The first kind of restriction is usually achieved by fixing a set © of execution fragments before- 
hand and requiring that all the probabilistic execution fragments H generated by an adversary 
satisfy Qy C O. We call the corresponding adversary schema an execution-based adversary 
schema. The second kind of restriction is achieved by imposing a correlation on the choices of 
an adversary on different inputs. We call the corresponding adversary schema an adversaries 
schema with partial on-line information. 


Example 5.1.5 An example of an execution-based adversary schema is the set of fair adver- 
saries for n processes running in parallel. In this case © is the set of execution fragments of 
the composite system where each process performs infinitely many transitions. An example of 
an adversary schema with partial on-line information is the set of oblivious adversaries for the 
toy resource allocation protocol. Execution-based adversary schemas and adversary schemas 
with partial on-line information can be combined together. An example of an execution-based 
adversary schema with partial on-line information is the set of fair and oblivious adversaries 
for the toy resource protocol (cf. Example 5.6.2). a 


Exacution-based adversaries and event schemas give us a good basis to study the relationship 
between deterministic and general adversaries. Roughly speaking, and adversary is determin- 
istic if it does not use randomness in its choices. Then the question is the following: “does 
randomness add power to an adversary?” The answer in general is “yes”; however, there are 
several situations of practical relevance where randomness does not add any power to an ad- 
versary. In particular, we show that randomization does not add any power when dealing with 
finitely satisfiable execution-based event schemas in two scenarios: execution-based adversary 
schemas and adversary schemas with partial on-line information. 


5.2 Adversaries and Adversary Schemas 


An adversary, also called a scheduler, for a probabilistic automaton M is a function A that 
takes a finite execution fragment a of MW and returns a combined transition of M that leaves 
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from Istate(a). Formally, 
A : frag*(M) — Probs(ctrans(M)) 
such that if A(a@) = (s,P), then s = Istate(a). 


An adversary is deterministic if it returns either transitions of M or pairs of the form 
(s,D(6)), ie., the next transition is chosen deterministically. Denote the set of adversaries 
and deterministic adversaries for a probabilistic automaton M by Advs(M) and DAdvs(M), 
respectively. We introduce deterministic adversaries explicitly because most of the existing 
randomized algorithms are analized against deterministic adversaries. In Section 5.7 we study 
the connections between deterministic adversaries and general adversaries. 

As we have noted already, the correctness of an algorithm may be based on some specific 
assumptions on the scheduling policy that is used. Thus, in general, we are interested only in 
some of the adversaries of Advs(M). We call a subset of Advs(.M) an adversary schema, and 
we use Advs to denote a generic adversary schema. Section 5.6 describes in more detail possible 
ways to specify an adversary schema. 


5.2.1 Application of an Adversary to a Finite Execution Fragment 


The interaction of an adversary A with a probabilistic automaton M leads to a probabilistic 
execution fragment, where the transition enabled from each state is the transition chosen by 
A. Given a finite execution fragment a of M, the probabilistic execution of M under A with 
starting condition a, denoted by prexec( M, A, a), is the unique probabilistic execution fragment 
HT of M such that 


1. start(H) = {a}, and 
2. for each state g of H, the transition tri? is ¢q~ A(q). 
Condition 2 ensures that the transition enabled from every state g of H is the transition chosen 
by A. It is a simple inductive argument to show that A is well defined. 
5.2.2 Application of an Adversary to a Finite Probabilistic Execution Frag- 
ment 


From the theoretical point of view, we can generalize the idea of the interaction between an 
adversary and a probabilistic automaton by assuming that the start condition is a finite prob- 
abilistic execution fragment of M. In this case the adversary works from all the points of 
extension of the starting condition. The resulting probabilistic execution fragment should be 
an extension of the starting condition. Formally, if H is a finite probabilistic execution fragment 
of M, then the probabilistic execution of M under A with starting condition H, denoted by 
prevec( M,.A, H), is the unique probabilistic execution fragment H’ of M such that 


1. start(H’) = start(H), and 


2. for each state g of H’, if g is a state of H, then tr is 
p (iri! t acts(H)) +(1—p) (47 AQ), 
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Figure 5-2: An example of the action of an adversary on a probabilistic execution fragment. 


where 


Pr[Cq] 


oe Pr [Cy] 


PF acts(H)], 


and if g is not a state of H, then ae is q~ A(q). 


Once again, it is a simple inductive argument to show that H’ is well defined. 


Example 5.2.1 (Extension of a finite probabilistic execution fragment) Before prov- 
ing that H’ is an extension of H, we describe in more detail how the definition above works. 
The difficult case is for those states g of H’ that are also states of H. Consider the example of 
Figure 5-2. Let A choose gg —> ¢ on input qo, choose q¢ = g2 on input g, and choose 6 on all 
other inputs. The probabilistic execution fragment H’ of Figure 5-2 is the result of the action 
of A on the probabilistic execution fragment H of Figure 5-2. In H’ there are two ways to reach 
gq: one way is by means of transitions of H, and the other way is by means of transitions due 
to A that originate from qo. Thus, a fraction of the probability of reaching g in H’ is due to 
Hf, while another fraction is due to the effect of A on H. The weight with which the transition 
ne is considered in H’ is the first fraction of the probability of reaching g, which is expressed 
by Pry[Cq|/PH[C,]. In our example the fraction is 1/2. However, in our example the transition 
bel? may also leads to 6 with probability 1/2, and the part of inl that leads to 6 should be 
handled by A. For this reason in the left term of the definition of ic we discard 6 from ind 
and we add a multiplicative factor PH acts( H)| to the weight. Thus, in our example, three 
quarters of the transition leaving from q in H’ are controlled by A. Note that the probability 
of reaching q, from qo is the same in H and #’. | 


Proposition 5.2.1 Let M be a probabilistic automaton, and let A be an adversary for M. 
Then, for each finite probabilistic execution fragment H of M, the probabilistic execution frag- 
ment generated by A from H is an extension of H, i.e., 


H < prexec(M,A, H). 
Proof. Denote prerec(M,A,H) by H’. We need to prove that for each state g of H, 
Pa[Cg] < Py |[Cq]- (5.7) 


If g is the start state of H, then q is also the start state of H’, and (5.7) is satisfied trivially. 
Consider now a state gas of H that is not the start state of H. Then q is a state of H. 
From the definition of the probability of a cone, 


Pip [Cyas] = PrlCy] PF ((a, qa). (5.8) 
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From the definition of tri? " 


' _ Pu(C] 
PI \(a,qas)] = Pufcyee aoa) + (: _ 


Feta el acs a) Pacayl(4, qas)|. (5.9) 
Here we have also simplified the expression P# [acts(H)] in the first term as we did in the proof 
of Proposition 4.3.5 (Expressions (4.33) and (4.34)). We will not mention this simplification 
any more in the thesis. 
If we remove the second term from the right expression of Equation (5.9), turning Equa- 
tion (5.9) into an inequality, we obtain 
Py[Cq| 


P#'T(a,qas)] > Pyle ee eae) (5.10) 


By using (5.10) in (5.8), and simplifying the factor Pr [C,], we obtain 


Prp[Cas] > PulCq) PF ((a, gas). (5.11) 


q 


The right part of (5.11) is Py[Cgas]. Thus, we conclude 


Py [Cras] > PulCrasl- (5.12) 


5.3 Event Schemas 


In the informal description of a probabilistic statement we said that we need a rule to associate 
an event with each probabilistic execution fragment. This is the purpose of an event schema. 
An event schema for a probabilistic automaton M, denoted by e, is a function that associates an 
event of Fy with each probabilistic execution fragment H of M. An event schema e is finitely 
satisfiable iff for each probabilistic execution fragment H the event e( 1) is finitely satisfiable. 
Union, intersection and complementation of event schemas are defined pointwise. Similarly, 
conditional event schemas are defined pointwise. 

The best way to think of an event schema is just as a rule to associate an event with 
each probabilistic execution fragment. Although in most of the practical cases the rule can be 
specified by a set of executions (cf. Section 5.3.2), part of our results do not depend on the 
actual rule, and thus they would hold even if for some reason in the future we need to study 
different rules. Moreover, event schemas allow us to simplify the notation all over. 


5.38.1 Concatenation of Event Schemas 


If e is a finitely satisfiable event schema, i.e., for each probabilistic execution fragment H the 
event e( H) can be expressed as a union of cones, then it means that in every execution of e(H ) 
it is possible to identify a finite point where the property denoted by e is satisfied. Sometimes 
we may be interested in checking whether a different property, expressed by another event 
schema, is satisfied eventually once the property expressed by e is satisfied. That is, we want 
to concatenate two event schemas. 
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Formally, let e,,e2 be two event schemas for a probabilistic automaton M where e, is finitely 
satisfiable, and let Cones be a function that associates a set Cones(H) with each probabilistic 
execution fragment H of M such that Cones(#H) is a characterization of e;(H) as a union of 
disjoint cones, i.e., e1(H) = Ugecones(H)Cq, and for each qm, q2 € Cones(H), if qi A qo, then 
Cy, Cg, = 9. Informally, Cones(H) identifies the points where the event denoted by e;(/) is 
satisfied, also called points of satisfaction. 

The concatenation €1 ©Cones €2 Of ey and eg via Cones is the function e such that, for each 
probabilistic execution fragment H of M, 


e(H) = LJ e2( Aq). (5.13) 
qg€ Cones(H) 


Proposition 5.3.1 The concatenation of two event schemas is an event schema. That is, if 
€ = €1 OCones €2, then € is an event schema. 


Proof. Consider a probabilistic execution fragment H. From Proposition 4.2.11 each set 
€2(H|q) is an event of Fy. From the closure of a o-field under countable union, e(/) is an 
event of Fy. | 


Proposition 5.3.2 Py[e1 ocones €2(H)| = oye cones(H) PHICa] Pu |qlea( Hq). 


Proof. Since Cones(H) represents a collection of disjoint cones, from (5.13) we obtain 


Pyles Cones €2(H)] = S- Pyie2( H|q)). (5.14) 
q€ Cones(H) 


From Proposition 4.2.11, for each g € Cones(H) 
Pyleo(H|q)] = PalCq]Pr|qlea(Z|4)I- (5.15) 


By substituting (5.15) in (5.14) we obtain the desired result. = 


5.38.2 Execution-Based Event Schemas 


Our definition of an event schema is very general; on the other hand, most of the existing 
work on randomized algorithms is based on a very simple rule to associate an event with each 
probabilistic execution. Namely, a set © of execution fragments of M is chosen beforehand, and 
then, given a probabilistic execution fragment H, the event associated with H is the 07 Oy. 
We call this class of event schemas execution-based. We have chosen to give a more general 
definition of an event schema for two main reasons: 


1. The concatenation Theorem of Section 5.4.1 (Theorem 5.4.2) does not rely on the fact that 
an event schema is execution-based, but rather on the fact that it is finitely satisfiable. 
Thus, if in the future some different kinds of event schemas will become relevant, here we 
have already the machinery to deal with them. 


2. The event schemas that we use later to define a progress statement (cf. Section 5.5) are 
not execution-based according to the informal description given above. Specifically, the 
start state of a probabilistic execution fragment of M is a finite execution fragment of 
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M,i.e., it contains some history of M, and such history is not considered in determining 
whether there is some progress. On the other hand, it is plausible that sometimes we 
want to consider also the history encoded in the start state of a probabilistic execution 
fragment. Thus, the more general definition of an event schema still helps. 


Nevertheless, execution-based adversary schemas are easier to understand and enjoy properties 
that do not hold for general adversary schemas (cf. Section 5.7). For this reason we give 
a formal definition of an execution-based adversary schema, where we also assume that the 
history encoded in the start state of a probabilistic execution fragment is eliminated. 

Let © be a set of extended execution fragments of 7. An event schema e for a probabilistic 
automaton M is ©-based iff for each probabilistic execution fragment H of M, e(H) = ON 
(Qyeg!). An event schema e for a probabilistic automaton M is execution-based iff there exists 
a set © of extended execution fragments of M such that e is O-based. 


5.4 Probabilistic Statements 


Given a probabilistic automaton M, an event schema e, an adversary A, and a finite execution 
fragment a, it is possible to compute the probability Ppreree(m,A,a)le(prerec( M,.A, a))] of the 
event denoted by e when M starts from a and interacts with A. As a notational convention, 
we abbreviate the expression above by Pyy,4,.[e]. Moreover, when M is clear from the context 
we write P4 |e], and we write Pyle] if M has a unique start state and a is chosen to be the 
start state of M. 

We now have all the machincery necessary to define a probabilistic statement. A probabilistic 
statement for a probabilistic automaton M is an expression of the form Pr4ds,o(e) R p, where 
Advs is an adversary schema of M, © is a set of starting conditions, i.e., a set of finite execution 
fragments of M, e is an event schema for M, and FR is a relation among =, <, and >. A 
probabilistic statement Pr4qys,o(e) R p is valid for M iff for each adversary A of Advs and each 
starting condition a of O, Pyle] R p, ie., 


Pradus,o(€) R P iff V Ae Advs Voe@ Pao [€] R Pp. (5.16) 
Proposition 5.4.1 Some trivial properties of probabilistic statements are the following. 
1. If py R po then Prads,o(e) R pr implies Pradse(e) R po. 


2. If Advs, C Advsy and 01 C Og, then Pr4ds,,o,(e) R p implies Pradys,,o.(€) R p. | 


5.4.1 The Concatenation Theorem 


We now study an important property of probabilistic statements applied to the concatenation 
of event schemas. Informally, we would like to derive properties of the concatenation of two 
event schemas from properties of the event schemas themselves. The idea that we want to 
capture is expressed by the sentence below and is formalized in Theorem 5.4.2. 


“Tf e, is satisfied with probability at least p,, and from every point of satisfaction of 
€1, €2 1s satisfied with probability at least pz, then the concatenation of e, and eg is 
satisfied with probability at least py po.” 
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Theorem 5.4.2 Consider a probabilistic automaton M. Let 

1. Pr4dvs,o(€1) R pr and, 

2. for each A € Advs, q € O, let Pr jays Cones(prevec(M,A,q))(€2) R P2- 
Then, Pr advs,o(€1 Cones €2) R Pipa: 


Proof. Consider an adversary A € Advs and any finite execution fragment gq € ©. Let 
H = prexec(M,A,q). From Proposition 5.3.2, 


Priet ° Cones €2(H)] = S- Py[Cy\ Pag le2 Aq). (5.17) 
q'€ Cones(H) 


Consider an element q' of Cones(H). It is a simple inductive argument to show that 
H\q' = prevec(M, A, q’). (5.18) 


Thus, from our second hypothesis, 


Prjqle2(1|q')] R pr. (5.19) 
By substituting (5.19) in (5.17), we obtain 
Pret Cones €2(H)] R po S- Py [Cai]. (5.20) 


q'€ Cones(e1(H)) 


By using the fact that Cones(H) is a characterization of e;(H) as a disjoint union of cones, 
Equation (5.20) can be rewritten into 


Pyles OCones €2(1)|] R poPulei(H)]. (5.21) 
From the first hypothesis, Py[e1()] R pi; therefore, from Proposition 5.4.1, 

Przle1 Cones €2(H)] R pipe: (5.22) 
This completes the proof. | 


5.5 Progress Statements 


In this section we give examples of probabilistic statements, which we call progress statements, 
that play an important role in the analysis of algorithms. Progress statements are formaliza- 
tions of statements that are used generally for the informal analysis of randomized algorithms; 
however, many other statements can be defined depending on specific applications. We show 
also how to derive complex statements by concatenating several simple statements. 
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5.5.1 Progress Statements with States 


Let U and U’ be sets of states of a probabilistic automaton M. A common informal statement 
is the following. 


“Whenever the system is in a state of U, then, under any adversary A of Advs, the 
probability that a state of U' is reached is at least p.” 


The probability p is usually 1. In this thesis we consider the more general statement where p 
is required only to be greater than 0. We represent the statement concisely by writing 


U Adv U'," (5.23) 


where Advs is an adversary schema. We call (5.23) a progress statement since, if we view U' as 
a better condition than U, then (5.23) states that from U it is possible to have some progress 
with probability at least p. The reader familiar with UNITY [CM88] may note that a progress 
statement is a probabilistic generalization of the leads-to operator of UNITY. 

Let us concentrate on the formal meaning of (5.23). Let ez be an event schema that given 
a probabilistic execution fragment H returns the set of extended executions a of Qy such that 
a state of U’ is reached in avgé! (recall that q@ is the start state of H). Then (5.23) is the 
probabilistic statement 


Pradus,u(eu) > p. (5.24) 


Note that the starting conditions of statement (5.24) are just states of M, i.e., they do not 
contain any past history of M except for the current state. This is because when we reason 
informally about algorithms we do not talk usually about the past history of a system. However, 
if we want to concatenate two progress statements according to Theorem 5.4.2, then we need to 
consider the past history explicitly, and thus a better probabilistic statement for (5.23) would 
be 


Pradus,Oy(eu') = P, (5.25) 


where Oy is the set of finite execution fragments of MM whose last state is a state of U. So, why 
can we, and indeed do people, avoid to deal with the past history explicitly? The point is that 
(5.24) and (5.25) are equivalent for most of the adversary schemas that are normally used. 


5.5.2 Finite History Insensitivity 


An adversary schema Advs for a probabilistic automaton M is finite-history-insensitive iff 
for each adversary A of Advs and each finite execution fragment a of M, there exists an 
adversary A’ of Advs such that for each execution fragment a’ of M with fstate(a’) = Istate(a), 
A'(a’) = A(a~ a’). In other words, A’ does even though A’ does not know the finite history a. 


Lemma 5.5.1 Let Advs be a finite-history-insensitive adversary schema for a probabilistic au- 
tomaton M. Then (5.24) and (5.25) are equivalent probabilistic statements. 
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Proof. From Proposition 5.4.1, since U C Oy, Statement (5.25) implies Statement (5.24) 
trivially. Conversely, suppose that Statement (5.24) is valid. Consider an adversary A of Advs, 
and consider an element ¢ of Oy. Let A, be an adversary of Advs such that for each execution 
fragment q' of M with fstate(q') = Istate(q), Ag(q') = A(q~ '). We know that A, exists since 
Advs is finite-history-insensitive. It is a simple inductive argument to show that 


prexec( M, Ag, lstate(q)) = prexec( M,.A, q)rq. (5.26) 
Moreover, 
Porevee(M,A,q)|Ca] =. (5.27) 


From the definition of ey, since the start state of prexec( M,A,¢q) is q, 
eu(prexec( M, Ag, lstate(q))) = eu'(prexec( M, A, q))rq. (5.28) 


Thus, from Proposition 4.2.12 and (5.27), 


Pagleu'] = Pa, lstate(q)leu'|- (5.29) 
From hypothesis, 

PA, lstate(q) lew") > Pp, (5.30) 
and thus, from (5.29), Pa 4lev’] > p. This shows that Statement (5.25) is valid. a 


5.5.3 The Concatenation Theorem 


We now start to compose (simple) progress statements to derive other (more complex) progress 
statements. This allows us to decompose a complex problems into simpler problems that can be 
solved separately. The examples of Chapter 6 contain explicit use of the concatenation theorem 
of this section. 

Suppose that from U we can reach U’ with probability at least p, and that from U’ we 
can reach U" with probability at least p’. Then, it is reasonable that from U we can reach U” 
with probability at least pp’. This result is an instantiation of the concatenation theorem of 
Section 5.4.1. 


Theorem 5.5.2 Let Aduvs be a finite-history-insensitive adversary schema. Then, 


U Ades U' and U' Sy Ados U" imply U Spr Ades U". 


Proof. Consider the event schemas ery and ey. Let Cones be the function that associates 
with each probabilistic execution fragment H the set 


Cones(H) = {q| Istate(qogo) € U', Ag! <(qoqo) Istate(q') € U"}. (5.31) 


It is easy to check that Cones(H) is a characterization of ez as a disjoint union of cones. Then, 
directly from the definitions, for each execution fragment A, 


€U' OCones eun(H) Cc eun(H). (5.32) 
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Informally, the left expression represents the property of reaching a state of U” passing through 
a state of U’, while the right expression represents the property of reaching a state of U” without 
passing necessarily through a state of U’. 

From Lemma 5.5.1, for each probabilistic execution fragment A, each adversary A of Advs, 
and each element ¢ of Cones(H), since Istate(q) € U’, 


Pagleun] > vp’. (5.33) 
From hypothesis, (5.33), and Theorem 5.4.2 (concatenation of two event schemas), 

Pradvs,U(€U! Cones CU") > pp’ (5.34) 
From (5.32) and (5.34), 


Pradvs,u(eu") > pp’. (5.35) 
This shows that U —4as U". | 
PP 


Proposition 5.5.3 Other trivial properties of progress statements are the following. 


1.0—U. 
1 


2. If Uy wR Uj and U2 > Us, then U,UU, —=  U; UU. a 


min(p1,p2) 


5.5.4 Progress Statements with Actions 


Progress statements can be formulated also in terms of actions rather than states. Thus, if V 
is a set of actions, we could write 


U Adv V (5.36) 


meaning that starting from any state of U and under any adversary of Advs, with probability at 
least p an action from V occurs. Formally, let ey be an event schema that given a probabilistic 
execution fragment H returns the set of executions a of Qy such that an action from V occurs 
in avg. Then (5.36) is the probabilistic statement 


Pradusu(ev) = p. (5.37) 
Similarly, we can change the left side of a progress statement. Thus, we can write 


V ~p Adus U (5.38) 


meaning that starting from any point where an action from V occurred and no state of U is 
reached after the last occurrence of an action from V, a state of U is reached with probability 
at least p. In other words, after an action from V occurs, no matter what the system has 
done, a state of U is reached with probability at least p. Formally, let Ov.z7 be the set of finite 
execution fragments of M where an action from V occurs and no state of U occurs after the 
last occurrence of an action from V. Then (5.38) is the probabilistic statement 


Pradus,Oyy (eu) = P. (5.39) 
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Finally, we can consider statements involving only sets of actions. Thus, the meaning of 
Vv Adv V’ would be the probabilistic statement 


Pradvs,Oy yi (ev) 2 Ps (5.40) 


where Oy,v is the set of finite execution fragments of M where an action from V occurs and 
no action from V’ occurs after the last occurrence of an action from V. 
The concatenation theorem extendeds easily to the new kinds of progress statements. 


Theorem 5.5.4 Let Advs be a finite-history-insensitive adversary schema, and let X,X' and 
X" be three sets, each one consisting either of actions of M only or states of M only. Then, 


X —4dys X! and X! —+4ays X" imply X ——pans X". 
P1 p2 P1p2 


Proof. This proof is similar to the proof of Theorem 5.5.2, and thus it is left to the reader. 
Observe that finite-history-insensitivity is not necessary if X’ is a set of actions. | 


5.5.5 Progress Statements with Probability 1 


Usually we are interested in progress properties that hold with probability 1. A useful result is 
that in most cases progress with probability 1 can be derived from progress with any probability 
psuch that 0 < p< 1. Specifically, under the condition that an adversary never chooses 6 when 
the left side of a given progress statement is satisfied and the right side of the same progress 
statement is not satisfied, 


1. if the left element of the progress statement is a set of actions, then progress is achieved 
with probability 1; 


2. if the left element of the progress statement is a set of states U, the adversary schema is 
finite-history-insensitive, and the system remains in a state of U unless the right side of 
the statement is satisfied, then progress is achieved with probability 1. 


Proposition 5.5.5 Suppose that V Ades X, and suppose that 6 ¢ Q4(q) for each adversary 
A of Advs and each element ¢ of Ov.x. Then V rAdus XxX. 


Proof. We give the proof for the case where X is a set of states. The other proof is similar. 
Denote X by U. 

Consider an element go of Ov, and an adversary A of Advs. Let H be prexec(M,.A, qo), 
and let p’ = Pyleu(H)]. We know from hypothesis that p’ > p. Suppose by contradiction that 
p' <1. Let © be the set of finite execution fragments q of M such that qo < q, Istate(q) € U, 
and no state of U occurs in any proper prefix of gogo. Then © is a characterization of ey(H) 
as a union of disjoint cones. Thus, 

Pyleu(H)] = S- Pr{Cq]. (5.41) 
qeO 
Let € be any real number such that 0 < € < p’. Then, from (5.41) and the definition of p’, it is 
possible to find a natural number &, such that. 


S> PxlC,) > (n'- 0. (5.42) 
q€Ollal<ke 
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Let ©, be the set of states g of H such that |g] = &. and no prefix of ¢ is in ©. That is, O, is 
the set of states of H of length k, that are not within any cone Cy of ey(H) where |q| < ke. 
Equation (5.41) can be rewritten as 


Pulev(#)] = ( 3 mut) + ( 3 PulPev HN) | (5.43) 
q€O|lq|<he qe9e 

Observe that for each state ¢ of O,, since a state of U’ is not reached yet, ¢ is an element of Oy. 

Moreover, prevec(M,A,q) = H|q (simple inductive argument). Thus, from Proposition 4.2.11 

and hypothesis, Pyler()|Cy] > p, and (5.43) can be rewritten into 


Puyleu()| 2 ( » onic) + (x Pics) (5.44) 


q€O|lq|<ke qe 
Observe that )jce|q<k. PHlCy] + Xyeo. Px[Cy] = 1. This follows from the fact that if a state 
q of H does not have any prefix in O, then g € Oy,x, which in turn means that 6 ¢ QF. In 


other words, in H it is not possible to stop before reaching either a state of {q € © | |q| < ke} 
or a state of O,. Thus, by using (5.42) in (5.44) we obtain 


PulevUD)] > (p' = 9) + (= (v= pp. (5.45) 
After simple algebraic manipulations, Equation (5.45) can be rewritten into 
Prleu(H)] = p+ pl — p') — (1 — p). (5.46) 


If we choose € such that 0 < € < p(1—p’)/(1—p), which exists since p’ < 1, then Equation (5.46) 
shows that Pyler(H)] > p’. This contradicts the fact that p’< 1. Thus, Pyler(H)]/=1. 


For the next proposition we define the statement U Unless X, where U is a set of states and X 
is either a set of states only or a set of actions only. The statement is true for a probabilistic 
automaton M iff for each transition (s,P) of M, if s ¢ U — X then for each (a, s’) € 2 either 
aéX,ors’ €UUX. That is, once in U, the probabilistic automaton M remains in U until 
the condition expressed by X is satisfied. 


Proposition 5.5.6 Suppose that U Adv X, U Unless X, Advs is finite-history-insensitive, 
and 6 ¢€ Qa(s) for each adversary A of Advs and each state s of U. Then, U Adu Xx. 


Proof. This proof is similar to the proof of Proposition 5.5.5. The main difference is that the 
passage from Equation (5.43) to Equation (5.44) is justified by using finite-history-insensitivity 
as in the proof of Proposition 5.5.1. | 


5.6 Adversaries with Restricted Power 


In Section 5.2 we have defined adversary schemas to reduce the power of an adversary; however, 
we have not described any method to specify how the power of an adversary is reduced. In 
this section we show two methods to reduce the power of an adversary. The first method, 
which is the most commonly used, reduces the kind of choices that an adversary can make; 
the second method, which is used in informal arguments but is rarely formalized, reduces the 
on-line information used by an adversary to make a choice. The two specification methods are 
used in Section 5.7 to study the relationship between deterministic and randomized adversaries. 
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5.6.1 Execution-Based Adversary Schemas 


If n processes run in parallel, then a common requirement of a scheduler is to be fair to all the 
processes. This means that whenever an adversary resolves the nondeterminism and leads to 
a probabilistic execution fragment H, in all the executions of Qy each one of the n processes 
performs infinitely many transitions. More generally, a set © of extended execution fragments 
of M is set beforehand, and then an adversary is required to lead only to probabilistic execution 
fragments whose corresponding sample space is a subset of O. 

Formally, let O be a set of extended execution fragments of MW. Let Advse be the set of 
adversaries A such that for each finite execution fragment ¢ of M, OQpreeee(M,Ayg) G O. Then 
Advse is called O-based. An adversary schema Advs is execution-based iff there exists a set O 
of extended execution fragments of M such that Advs is O-based. 

The notion of finite-history-insensitivity can be reformulated easily for execution-based ad- 
versary schemas. Define © to be finite-history-insensitive iff for each extended execution frag- 
ment a of M and each finite execution fragment a’ of M such that Istate(a’) = fstate(a), if 
a’~a€O then a € O. It is easy to verify that if © is finite-history-insensitive, then Advse is 
finite-history-insensitive. 


5.6.2 Adversaries with Partial On-Line Information 


Sometimes, like in the case of the toy resource allocation protocol, an adversary cannot base 
its choices on the whole history of a system if we want to guarantee progress. In other words, 
some part of the history is not visible to the adversary. 


Example 5.6.1 (Off-line scheduler) The simplest kind of adversary for n processes that run 
in parallel is an adversary that fixes in advance the order in which the processes are scheduled. 
This is usually called an off-line scheduler or an oblivious adversary. Thus, at each point a 
the next transition to be scheduled depends only on the ordered sequence of processes that are 
scheduled in a. 

To be more precise, the transition scheduled by the adversary depends also on the state that 
is reached by a, i.e., Istate(a), since a specific process may enable different transitions from 
different states. This means that if a, and a2 are equivalent in terms of the ordered sequence 
of processes that are scheduled, the oblivious constraint says only that the transitions chosen 
by the adversary in a; and a2 must be correlated, i.e., they must be transitions of the same 
process. a 


The formal definition of an adversary with partial on-line information for a probabilistic au- 
tomaton M is given by specifying two objects: 


1. an equivalence relation that specifies for what finite execution fragments of M the choices 
of an adversary must be correlated; 


2. a collection of correlation functions that specify how the transitions chosen by an adver- 
sary must be correlated. 


Let = be an equivalence relation between finite execution fragments of M, and let F be a 
family of functions parameterized over pairs of equivalent execution fragments. Each function 
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faa takes a combined transition of M leaving from Istate(a) and returns a combined transition 
of M leaving from Istate(a’) such that 


1. fetal faa (tr)) = tr; 
2. foot ier pitti) = Vier Pifaat (tri). 


The pair (=, F’) is called an oblivious relation. An adversary A is oblivious relative to (=, F’) iff 
for each pair of equivalent execution fragments of M,a =a’, A(a’) = faas(A(a@)). An adversary 
schema, Advs is said to be with partial on-line information iff there exists an oblivious relation 
(=, F’) such that Advs is the set of adversaries for M that are oblivious relative to (=, fF’). 

Condition 1 is used to guarantee that there are oblivious adversaries relative to (=, fF’); 
Condition 2 is more technical and is used to guarantee that there are oblivious adversaries 
relative to (=, F’) that do not use randomization in their choices. Condition 2 is needed mainly 
to prove some of the results of Section 5.7. 

Adversaries with partial on-line information and execution-based adversaries can be com- 
bined together easily. Thus, an adversary schema Advs is said to be execution-based and with 
partial on-line information iff there exists an execution-based adversary schema Advs’ and a 
pair (=, F’) such that Advs is the set of adversaries of Advs’ that are oblivious relative to (=, F). 


Example 5.6.2 (Adversaries for the toy-resource allocation protocol) The fair obliv- 
ious adversaries for the toy resource allocation protocol are an example of an execution-based 
adversary schema with partial on-line information. The set O is the set of executions of M,|| M2 
where both M4; and My, perform infinitely many transitions. Two finite execution fragments 
ay and a2 are equivalent iff the ordered sequences of the processes that perform a transition 
in a; and a2 are the same. Let ay = ag, and let, for 2 = 1,2, tr;, and tr; be the tran- 
sitions of M, and Mg, respectively, enabled from Istate(a;). Then fa,o.(tria) = tra, and 
foroz(tr1,2) = tra. 

Another execution-based adversary schema with partial on-line information that works for 
the toy resource allocation protocol is obtained by weakening the equivalence relation so that 
an adversary cannot see only those coins that have not been used yet, i.e., those coins that have 
been flipped but have not been used yet to check whether the chosen resource is free. | 


5.7 Deterministic versus Randomized Adversaries 


In our definition of an adversary we have allowed the use of randomness for the resolution of 
the nondeterminism in a probabilistic automaton M. This power that we give to an adversary 
corresponds to the possibility of combining transitions of M in the definition of a probabilistic 
execution fragment. From the formal point of view, randomized adversaries allow us to model a 
randomized environment and to state and prove the closure of probabilistic execution fragments 
under projection (Proposition 4.3.4). However, one question is still open: 


Are randomized adversaries more powerful than deterministic adversaries? 


That is, if an algorithm performs well under any deterministic adversary, does it perform well 
under any adversary as well, or are there any randomized adversaries that can degrade the 
performance of the algorithm? In this section we want to show that in practice randomization 
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does not add any power to an adversary. We say ”in practice” because it is easy to build 
examples where randomized adversaries are more powerful than deterministic adversaries, but 
those examples do not seem to be relevant in practice. 


Example 5.7.1 (Randomization adds power) Consider an event schema e that applied to 
a probabilistic execution fragment H returns Oy if H can be generated by a deterministic 
adversary, and returns 9 otherwise. Clearly, if M is a nontrivial probabilistic automaton, the 
probability of e is at least 1 under any deterministic adversary, while the probability of e can 
be 0 under some randomized adversary; thus, randomization adds power to the adversaries. 
However, it is unlikely that a realistic event schema has the structure of e. Another less 
pathological example appears in Section 5.7.2 (cf. Example 5.7.2). | 


We consider the class of execution-based event schemas, and we restrict our attention to the 
subclass of finitely satisfiable, execution-based event schemas. We show that randomization does 
not add any power for finitely satisfiable, execution-based event schemas under two scenarios: 
execution-based adversary schemas, and execution-based adversary schemas with partial on-line 
information. In the second case we need to be careful (cf. Example 5.7.2). 

Informally, a randomized adversary can be seen as a convex combination of deterministic 
adversaries, and thus a randomized adversary satisfies the same probability bounds of a deter- 
ministic adversary. However, there are uncountably many deterministic adversaries, and thus 
from the formal point of view some more careful analysis is necessary. 


5.7.1 Execution-Based Adversary Schemas 


Proposition 5.7.1 Let Advs be an execution-based adversary schema for M, and let Adusp 
be the set of deterministic adversaries of Advus. Let e be a finitely-satisfiable, execution-based, 
event schema for M. Then, for every set O of finite execution fragments of M, every probability 
p, and every relation R among <, =, >, Prads,o(e) R p tf Pradsp.o(e) R p- | 


In the rest of this section we prove Proposition 5.7.1. Informally, we show that each probabilistic 
execution fragment H generated by an adversary of Advs can be converted into two other 
probabilistic execution fragments H’ and H”, each one generated by some adversary of Advsp, 
such that Py [e(H')| < Pyle(H)] < Pyu[e(H”)]. Then, if R is < we use H”, and if R is > we 
use H’. 

An operation that is used heavily in the proof is called deterministic reduction. Let H bea 
probabilistic execution fragment of a probabilistic automaton M, and let g be a state of H. A 
probabilistic execution fragment H’ is said to be obtained from H by deterministic reduction 
of the transition enabled from q if H’ is obtained from H through the following two operations: 


1. Let ir? = ¢7 (jer pitri) where each p; is non-zero and each tr; is a transition of M. 
Then replace tr/? either with (q,D(6)) or with q@ tr;, under the restriction that (q,D(é)) 
can be chosen only if Ye, pj < 1. 


2. Remove all the states of H that become unreachable after ir? is replaced. 


Throughout the rest of this section we assume implicitly that whenever a probabilistic execution 
fragment is transformed, all the states that become unreachable are removed. 
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Lemma 5.7.2 Let Advs be an execution-based adversary schema for a probabilistic automaton 
M, and let H be a probabilistic execution fragment of M that is generated by some adversary 
of Advs. Let e be an execution-based event schema such that Py|e(H)| = p. Let q be a state 
of H. Then there exist two probabilistic execution fragments Fis Thigh? each one generated 


by an adversary of Advs, that are obtained from H by deterministic reduction of the transition 
enabled from q, and such that Paya [e( 7, )] <p, and Paya. le iign > p. 
ow ag 


Proof. Let tri be ¢7 (jer pitri), Where each tr; is either a transition of M or the pair 
(Istate(q),D(6)), each p; is greater than 0, and >;<;p; = 1. For each transition tr;, i € I, let 
H,, be obtained from H by replacing tri with q~ tr;. Observe that, since Advs is execution- 
based and H is generated by an adversary of Advs, H;,, is generated by an adversary of Advs. 
The probability of e(H) can be written as 


Pule(H)] = PulCg)Pule(H)ICq] + (= PulCg)) Pile ICD. (5.47) 
Observe that for each 7 € J, since H and H;,, differ only in the states having q as a prefix, 
Py[Cq] = Pu,,,[Cq]- Since e is execution-based, e(H ) NC, = e( Hy, Cg, and Pyle(H)AC,] = 


Pr, [e( Htr,) A Cg] (use conditional probability spaces and Theorem 3.1.2). Moreover, as it is 
shown below, Py[e(H)O Cy] = Vie piPu,,.le(H,) A Ca]. In fact, 


Pyle HOC] = PalCq] | PP PaleED Cae) + DD Psa.) Parle) |Cq'] | (5.48) 
(a,q’')EQF 
where we assume that Py[e(H)|C,s] is 0 whenever it is undefined. For each (a,q') of QY, 
PH(q,a)] = ier piPy [(a,q’)], and for each 7 such that (a,q’) € Qn, Pyle(H#)|Cy] = 
Pr,,,[e( Her; )|Cq’] (simply observe that Hed = H,,,rq'). Similarly, if 6 € OF, then PH] = 
ier piPy [6], and for each i such that 6 € ag, Pryle(H)|Cqs] = Pry,,, le Her, )|Cqs]- Thus, 
from (5.48), 


Pyle(H) Cy] = Yo iP ite, [C,] 
tel 


pi 


Her. 
[]Pri,leHirNCgl+ = D2 Pa (aa) Pa, [eH Cal |. (5-49) 
(a.q/)eM, 
which gives the desired equality 
Prle(H) 0 Cy] = 7 pi Pry, le(Hir,) 0 Ca)- (5.50) 
tel 
Thus, (5.47) can be rewritten into 


Pule(H)| = >> pi (Pir, [Ca] Pity, le Her:)|Ca] + 1 = Pity, (Col) Pitey, [eC Hers Cal) + (5-51) 


wel 
which becomes 
Pyle(H)) = So pi Pity, leer ))- (5.52) 
wel 
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If there exists an element 7 of I such that Pry, [e( Hi; )] = p, then fix H7,, and ison to be H¢,,. 
If there is no element ¢ of J such that Py,,,[e(He;)] = p, then it is enough to show that there 
are two elements 21,22 of J such that Pit, [e( Air, )] < p and Pity, [e( Air, )] > p, respectively. 
Assume by contradiction that for each element 7 of I, Py, [e(Ht;)] <p. Then, from (5.52), 
ier PiPHy,,[e( Hr, )] <p, which contradicts Py{e( H)] = p. Similarly, assume by contradiction 
that for each element ¢ of I, Py,,,[e(Htr;)] > p. Then, from (5.52), Vier PiPuy,, [e(Hur,)] > P, 
which contradicts Py[e(H)] = p again. = 


Lemma 5.7.3 Let Advs be an execution-based adversary schema for a probabilistic automaton 
M, and let H be a probabilistic execution fragment of M_ that is generated by some adversary 
of Adus. Let e be an execution-based event schema such that Py|e(H)| = p. Let d be a natural 
number, and let Uz be the set of states q of H such that |q| = d. Then there exist two probabilistic 
execution fragments Higw, Hhigh, each one generated by an adversary of Advs, that are obtained 
from H by deterministic reduction of the transitions enabled from the states of Ug, and such 


that Pr,,,,le(Hiow)| < p, and PH igh [e(Hhigh)] > p- 


Proof. From Lemma 5.7.2 we know that for each state g of Uz there are two probabilistic exe- 
cution fragments H/,,, and Thigh obtained from H by deterministic reduction of the transition 
enabled from q, such that Pye le Hiw)] < p, and Pare. [e(Hhign)] = p- Let Hiow be obtained 


from H by replacing the transition enabled from each state gq of Ug with the transition enabled 
from gin H7,,, and let Hyig, be obtained from H by replacing the transition enabled from each 
state g of Ug with the transition enabled from gq in Tigh Since Advs is execution-based and 
all the involved probabilistic execution fragments are generated by an adversary of Advs, then 
Hyg, and Hj.y are generated by an adversary of Advs. Since e is execution-based, for each 
state gq of Ug, Pr,,,, [e(Hiow) A Cg] = Pra le(H ow) Cy]. Thus, 


PH wlel e( Hiow)| =») Pry wlC alPue le (H low IC al- (5.53) 
qeUg 
Observe that, for each state g of Ua, the difference between the probability of e(H) and the 
probability of e(H7|,,) is determined by the subcones of Cy. Thus, 


Pi,,,le(Hiow)| < S> Purl H)|C). (5.54) 

qeUa 
The right side of (5.54) is Py[e(H)], which is p. In a similar way it is possible to show that 
Pry ignl€( Lrigh)| 2 P- 7 


Now we use the fact that e is finitely satisfiable. For each probabilistic execution fragment H 
of M, let Can(e(H)) the set of minimal elements of {¢ € states(H)|C, C e(H)}U{q6 | qe 
states(H),Cgs C e(H)}. Then, Can(e(H)) is a characterization of e( 1) as a union of disjoint 
cones. For each natural number d, let efd be the function that given a probabilistic execution 
fragment H returns the set Uge Can(e(H))|lal<dy 


Lemma 5.7.4 Lete be an execution-based, finitely satisfiable, event schema for a probabilistic 
automaton M, and let d,d' be two natural numbers such that d < d'. Then, for each probabilistic 
execution fragment H, Pyletd(H)] < Pyletd'(H)] < Pyle()). 
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Proof. Follows trivially from the definitions. | 


Lemma 5.7.5 Let e be an execution-based, finitely satisfiable, event schema for a probabilistic 
automaton M, and let d be a natural number. Let H be a probabilistic execution fragment H 
of M, and let H’ be obtained from H by reducing deterministically any collection of states of 
length greater than d. Then, Py[etd(H)] < Py fetd( 1’). 


Proof. Just observe that for each g € Can(e(H)) such that |q| < d there is a q’ € Can(e(H’)) 
such that q’ < q, and that for each state g of H such that |¢| < d, PH[Cg] = Pu [Cy]. = 


Lemma 5.7.6 Let Advs be an execution-based adversary schema for a probabilistic automaton 
M, and let H be a probabilistic execution fragment of M that is generated by some adversary 
of Adus. Let e be an execution-based, finitely satisfiable event schema such that Py|e(H)]| = p. 
Then there exists a probabilistic execution fragment H', generated by a deterministic adversary 
of Advs, such that Py |e(H’)| < p. 


Proof. From Lemma 5.7.3 it is possible to find a sequence of probabilistic execution fragments 
(H;)i>o, where Hyp = H, each H;41 is obtained from H; by deterministically reducing all its 
transitions leaving from states of length i, and for each i, Py,,,[e(4i41)] < Pu,[e(H:)]. Let H’ 
be obtained from H by replacing the transition enabled from each state gq with the transition 
enabled from g in any H; such that |q| < i. It is immediate to check that H’ is generated by 
some deterministic adversary of Advs (every extended execution of 77; is an extended execution 
of Q77). 
Suppose by contradiction that Py [e(H’)] > p. Then there exists a level d such that 


Puiletd(H’)| > p. (5.55) 
For each d’ > d, let Ey be 
Ey = UJ cH. (5.56) 
g€ Can(e td’ (Ay ))lAgecan(etatH')) 0 <4 
Then, the following properties are valid. 
1. for each d’ > d, E', is an element of Fy. 
Ey is a union of cones of Fy. 
2. if d’<d", then Bg C Eq 


Consider an element g € Can(efd’(H,4)) such that there exists ag’ € Can(efd(H’)) such 
that q’ < q. Observe that, since Hy is obtained from Hy by deterministic reduction of 
states of length greater than d’, there exists a q” € Can(efd"(H4g)) such that q” < gq. 
Moreover, from the construction of H’, q’ < q". Thus, from (5.56), cH C Eg. Since 


dq’ <4, cH C Eq, and therefore, Ey C Eqn. 
3. e}d(H’) C Ua>aka. 


Consider an element a of efd(H’). Then, for each d’, a € e( Hy). Let q’ € Can(e( Ha)) 
such that q’ < a, and let d’ be |q’|. Then, there exists a q” € Can(efd’(Ha)) such that 
g’ <q <a, and thus a € Ey. 
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4, for each d! > d, Py, [etd Ha)] > Po [Ea). 


From the construction of H’, for each q such that |q| < d’, Pu, [cee] = Pr (CF). 


Moreover, if cH is used in the definition of Ey, then g € Can(efd'(Hy)). 


From 2 and 3, and from (5.55), there exists a value d’ such that Py [Ew] > p. From 4, 
Pr, letd'(Ha)| > p. From Lemma 5.7.4, Py,[e(Ha)] > p. This contradicts the fact that 
Pry [e td" Ha) <p. | 


To build a probabilistic execution fragment H’, generated by an adversary of Advsp, such that 
Py [e(H')| > p, we need to extend part of Lemmas 5.7.2 and 5.7.3. 


Lemma 5.7.7 Let Advs be an execution-based adversary schema for a probabilistic automaton 
M, and let H be a probabilistic execution fragment of M that is generated by some adversary of 
Advs. Let e be an execution-based, finitely-satisfiable, event schema. Let q be a state of H, and 
let d be a natural number such that Py|etd(H)|] = p. Then there exist a probabilistic execution 
fragment Tigh generated by an adversary of Advs, that is obtained from H by deterministic 
reduction of the transition enabled from q, such that Put, [e td Ho) > p. 


Proof. This proof is similar to the proof of Lemma 5.7.2, with the difference that the = sign 
of Equations (5.49), (5.50), (5.51), and (5.52), is changed into a <. In fact, in each one of the 
H,, some new cone of length at most d may appear. | 


Lemma 5.7.8 Let Advs be an execution-based adversary schema for a probabilistic automaton 
M, and let H be a probabilistic execution fragment of M_ that is generated by some adversary 
of Advs. Let e be an execution-based, finitely-satisfiable, event schema, and let d be a natural 
number such that Py[etd(H)| = p. Let d’ be a natural number, and let Ug be the set of states q 
of H such that |q| = d’. Then there exist a probabilistic execution fragment Hhjg,, generated by 
an adversary of Advs, that differs from H only in that the transitions enabled from the states 
of Uz are deterministically reduced, such that PH igh [efd( Hhigh)] > p- 


Proof. This proof is similar to the proof of Lemma 5.7.3. In this case the arguments for the 
equation corresponding to Equation (5.54) is justified from the additional fact that Hyj9, may 
have more cone of depth at most d than H. | 


Lemma 5.7.9 Let Advs be an execution-based adversary schema for a probabilistic automaton 
M, and let H be a probabilistic execution fragment of M_ that is generated by some adversary 
of Adus. Let e be an execution-based, finitely-satisfiable, event schema such that Py|e(H)| > p. 
Then, there exists a probabilistic execution fragment H' of M, generated by a deterministic 
adversary of Advs, such that Pyle(H')] > p. 


Proof. Since Py[e(H)] > p and e(#) is a union of cones, there exists a natural number d such 
that Py[etd(H)] > p. From repeated applications of Lemma 5.7.8, one for each level d’ < d, 
there exists a probabilistic execution fragment H”, obtained from H by deterministic reduction 
of the transitions enabled from every state gq with |g| < d, such that Pyv[etd(H”)] > p. From 
Lemma 5.7.4, Pyule( H")| > p. Moreover, any probabilistic execution fragment H’” obtained 
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from H” by reducing deterministically transitions at depth greater than d (|g| > d) satisfies 
Pymletd(H"”)| > p, and thus Pym[e(H’)] > p. Hence, H' can be any probabilistic execution 
fragment obtained from H” by reducing deterministically all the transitions at depth greater 
than din any arbitrary way. It is easy to check that H’ is generated by a deterministic adversary 


of Advs. a 


Lemma 5.7.10 Let Advs be an execution-based adversary schema for a probabilistic automaton 
M, and let H be a probabilistic execution fragment of M that is generated by some adversary 
of Adus. Let e be an execution-based, finitely-satisfiable, event schema such that Py|e(H)]| > p. 
Then, there exists a probabilistic execution fragment H' of M, generated by a deterministic 
adversary of Advs, such that Py[e(H')] > p. 


Proof. If Py[e(H)] > p, then Lemma 5.7.9 suffices. If Py[e(H)] = p, then by Lemma 5.7.3 
it is possible to find a sequence of probabilistic execution fragments (H;)j>0, where Hyp = H, 
each fH;41 is obtained from H; by deterministically reducing all its ¢-level transitions, and 
for each 7, Puy,,,[e(Hi41)] = Pu,le(H,)]. If there exists a sequence (H;);>0 such that for 
some i, Py,[e(H;)] > p, then Lemma 5.7.9 suffices. Otherwise, consider the sequence of 
probabilistic execution fragments defined as follows: Hg = H and, for each 2, let d; be 
the level of H; such that Py,[efd;(H;)] > Pd j<i(1/2)". Let H;41 be obtained from re- 
peated applications of Lemma 5.7.8, till level d;, so that Py,,,[ehd:(Hi+1)] > Pd j<i(1/2)7". 
Note that Py,,,[e(Hi41)] = p, otherwise we can find a sequence (H;);>0 and an 7 such that 
Pu,,,le(Hi+1)] > p (simple argument by contradiction). Let H' be obtained from H by replac- 
ing the transition enabled from each state q with the transition enabled from g in any H; such 
that |q| < d;_,. It is easy to check that H’ is generated by an adversary of Advs. Suppose by 
contradiction that Py:[e(H')] = p' < p. Then, from the construction of the H;’s, there exists an 
isuch that p>,<;(1/2)'*1 > p’, and thus Py,,,[etd;(Hi41)] > p’. However, from the definition 
of H’, Py, letd:(Hi41)] = Pa[etd;(H')], and thus p’ < Pr[e(H’)], which contradicts the fact 
that Py [e(H')] = p’. = 


Proof of Proposition 5.7.1. Since Adusp C Advs, Prads,o(e) R p implies Praas,o(e) R p 
trivially. Conversely, suppose that Pr4ams,,e(e) R p, and let H be a probabilistic execution 
fragment, generated by an adversary of Advs, whose start state is in ©. We distinguish the 
following cases. 


1. Ris >. 
From Lemma 5.7.6, there is a probabilistic execution fragment H’, generated by an ad- 
versary of Advsp, whose start state is in O, and such that Py [e(H')] < Py[e(H)]. From 
hypothesis, Py:[e(H’)] > p. Thus, Py[e(H)] > p. 

2. Ris <, 


From Lemma 5.7.10, there is a probabilistic execution fragment H’, generated by an 
adversary of Advsp, whose start state is in ©, and such that Py:[e(H')] > Pyle(H)]. 
From hypothesis, Py [e(H')] < p. Thus, Pyle(H)] < p. 


3. Ris =. 
This follows by combining Items 1 and 2. | 
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5.7.2 Execution-Based Adversary Schemas with Partial On-Line Informa- 
tion 


Proposition 5.7.1 can be extended to adversary schemas that do not know all the past history 
of a system, i.e., to execution-based adversary schemas with partial on-line information. We 
need to impose a technical restriction, though, which is that an adversary should always be 
able to distinguish two execution fragments with a different length (cf. Example 5.7.2). The 
proof of the new result is a simple modification of the proof of Proposition 5.7.1. 


Proposition 5.7.11 Let (=, I’) be an oblivious relation such that for each pair ay = ag of 
equivalent execution fragment, a, and ag have the same length. Let Adus be an execution- 
based adversary schema with partial on-line information such that each adversary of Advs is 
oblivious relative to (=,F'), and let Advsp be the set of deterministic adversaries of Advs. 
Let e be a finitely-satisfiable, execution-based, event schema for M. Then, for every set O of 
finite execution fragments of M, every probability p, and every relation R among <, =, >, 


Pradvs,o(e) R p uf Pradsp,ol(e) R p. 


Proof. The proof is similar to the proof of Proposition 5.7.1. The main difference is in the 
proofs of Lemmas 5.7.2, 5.7.3 and 5.7.8, where equivalence classes of states rather than single 
states only must be considered. In these two proofs we use also the fact that equivalent execution 
fragments have the same length. The details of the proof are left to the reader. | 


Example 5.7.2 (Why length sensitivity) The requirement that an adversary should al- 
ways see the length of a probabilistic execution fragment seems to be artificial; however, ran- 
domized adversaries have more power in general if they cannot see the length of a probabilistic 
execution. Consider the probabilistic automaton M of Figure 5-3, and suppose that all the 
executions of M that end in states 51, 59,53, and sg are equivalent. Since for each state s; there 
is exactly one execution of M that ends in s;, we denote such an execution by g;. Let O be the 
set of extended executions aé of M such that Istate(a) does not enable any transition in M. 
For each state s; that enables some transition, let tr;,, be the transition that leaves from s; and 
goes upward, and let tr; be the transition that leaves from s; and goes downward. Then, for 
each pair 2,7 € {1,2,3,6},74 9, let foig; (tiv) = trju, and let faiq,(tria) = trja- 

Let Advs be the set of O-based adversaries for M that are oblivious relative to (=, F’), and 
let Advsp be the set of deterministic adversaries of Advs. Then, the statement {so} pawn 


{87,510} is valid, whereas the statement {so} Tain {87,519} is not valid, i.e., an adversary can 


use randomization to reduce the probability to reach states {7,519}. In fact, the probabilistic 
executions H, and H2 of Figure 5-3 are the only probabilistic executions of M that can be 
generated by the adversaries of Advsp, while Ho is generated by an adversary of Advs. The 
probability of reaching {87,519} in H, and Hy is 1/2, whereas the probability of reaching 
{87,510} in Ho is 1/4. | 


5.8 Probabilistic Statements without Adversaries 


The current literature on randomized distributed algorithms relies on the notion of an adversary, 
and for this reason all the definitions given in this chapter are based on adversaries. However, 
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Figure 5-3: Randomization adds power for some adversaries with partial on-line information. 


the key objects of the theory that we have presented are the probabilistic execution fragments of 
a probabilistic automaton, and not its adversaries. An adversary schema can be replaced by an 
arbitrary set of probabilistic execution fragments in the definition of a probabilistic statement, 
namely, the set of probabilistic execution fragments that the adversary schema can generate. In 
other words, an adversary schema can be seen as a useful tool to express a set of probabilistic 
execution fragments. 


5.9 Discussion 


Two objects that we have defined in this chapter and that do not appear anywhere in the 
literature are adversary schemas and event schemas. Both the objects are needed because, 
differently from existing work, in this thesis we identify several different rules to limit the 
power of an adversary and several different rules to associate an event with a probabilistic 
execution fragment, and thus we need some way to identify each rule. The best way to think 
of an adversary schema and of an event schema is as a way to denote the rule that is used to 
limit the power of an adversary and denote the rule that is used to associate an event with each 
probabilistic execution fragment. 

We have defined the classes of execution-based adversary schemas and execution-based 
event schemas, and we have proved that for finitely satisfiable execution-based event schemas 
randomization does not increase the power of an execution-based adversary schema, or of a 
class of execution-based adversary schemas with partial on-line information. These results are 
of practical importance because most of the known event schemas and adversary schemas of 
practical interest are execution-based. As a result, it is possible to verify the correctness of 
a randomized distributed algorithm by analyzing only the effect of deterministic adversaries, 
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which is easier than analyzing every adversary. A similar result is shown by Hart, Sharir and 
Pnueli [HSP83] for fair adversaries and almost-sure termination properties, i.e., properties that 
express the fact that under all fair adversaries the system reaches some fixed set of states 
with probability 1. Fair adversaries and termination events are expressible as execution-based 
adversary schemas and finitely satisfiable execution-based event schemas, respectively; thus, 
the result of Hart, Sharir and Pnueli is implied by our result. Hart, Sharir and Pnueli prove 
also that another class of adversaries is equivalent to the class of fair adversaries, namely, those 
adversaries that lead to fair executions with probability 1. The same result holds here as well; 
however, it is not clear under what conditions a similar result holds in general. 
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Chapter 6 


Direct Verication Proving a 
Property 


In this chapter we illustrate techniques to prove the validity of a probabilistic statement from 
scratch. The main technique, which is based on coin lemmas, consists of reducing the analysis of 
a property of a probabilistic automaton to the analysis of a property of an ordinary automaton. 
We illustrate the methodology by applying it to some existing randomized algorithms. 

Part of this chapter is based on joint work with Anna Pogosyants and Isaac Saias. Anna 
Pogosyants suggested us the coin event OCC (Section 6.2.3) as a generalization of other less 
elegant coin events that we had in mind and collaborated on the verification of the randomized 
algorithm for agreement of Ben-Or (Section 6.5). The verification of the randomized dining 
philosophers algorithm of Lehmann and Rabin (Section 6.3) is based on joint work with Nancy 
Lynch and Isaac Saias [L$594], and the verification of the randomized algorithm for agreement 
of Ben-Or is a formalization of a proof that appears in the book on distributed algorithms of 
Nancy Lynch [Lyn95]. 


6.1 How to Prove the Validity of a Probabilistic Statement 


In Chapter 5 we have defined formally what is a probabilistic statement and we have shown how 
it is possible to combine probabilistic statements to derive more complex properties. However, 
one question is left open: how do we prove the validity of a given probabilistic statement from 
scratch? 

The problem is not trivial: a property may rely on complicate global configurations of a 
system that depend on several separated random draws. Analyzing the exact probability of an 
event associated with a probabilistic execution fragment may be extremely hard. Fortunately, 
there are usually some key points, known to the designer of a system, where specific probabilistic 
choices lead to the desired property. In this chapter we formalize the idea above by introducing 
a collection of coin lemmas. The idea behind a coin lemma is the following. 


1. We define a mechanism to identify events of the kind “some specific probabilistic choices 
yield some specific results”. We call such events coin events since a common source of 
randomness is given by coin flips. 
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2. We prove a lower bound on the probability of the coin event that we identify. 


Then, the analysis of a probabilistic statement for a probabilistic automaton M proceeds as 
follows. 


1. We find a coin event that expresses the key intuition behind the property to be shown. 


2. We show that the coin event is a subevent of the event expressing the desired property, 
i.e., we show that whenever the coin event is satisfied, the desired property is satisfied as 
well. 


3. We use the lower bound on the probability of the coin event to obtain a lower bound on 
the probability of the desired property. 


Example 6.1.1 (Coin lemmas and the toy resource allocation protocol) Let us con- 
sider the toy resource allocation protocol of Chapter 5 again. One of the coin lemmas of 
this chapter states that if we fix any two separate coin flips (flipping of different coins) and 
we consider the event where the two coin flips yield different outcomes whenever they both 
occur, then, no matter how the nondeterminism is resolved, the considered event is satisfied 
with probability at least 1/2. On the other hand, if the first coin flip of M, after the first coin 
flip of M@> is different from the last coin flip of Mz before the first time M, checks its resource 
after flipping, then M4, succeeds in getting its resource. Thus, whenever the property above can 
be expressed as a coin event in a form suitable to the coin lemma above, we are guaranteed that 
M, eventually gets its resource with probability at least 1/2. It turns out that an adversary 
must be fair, oblivious and deterministic in order to be able to define the desired coin event (cf. 
Section 6.6). Our results about deterministic and randomized adversaries (Proposition 5.7.11) 
can then be used to remove the constraint that an adversary is deterministic. | 


We present a large collection of coin lemmas, and we illustrate their use via two main examples: 
Section 6.3 proves the correctness of the randomized Dining Philosophers algorithm of Lehmann 
and Rabin [LR81], and Section 6.5 proves the correctness of the randomized algorithm of Ben- 
Or for agreement in asynchronous networks in the presence of stopping faults [BO83]. At the 
end of the chapter we hint at another technique, called the partition technique, that departs 
considerably from the coin lemmas and that is necessary to prove stronger claims about the toy 
resource allocation protocol. We leave to further work a deeper study of this other technique. 


6.2 Some Simple Coin Lemmas 


In this section we present some simple coin lemmas where we use actions to identify the random 
draws of interest. Specifically, we study the following coin lemmas. 


1. First occurrence of an action. 


In this coin lemma we consider an action a and a set of states U, and we study the 
probability that either action a does not occur or the first occurrence of action a leads to 
a state of U. We show that this probability is at least the infimum of the probability of 
reaching a state of U over all the transitions of M that are labeled with action a. 
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As an example, action a can identify the process of flipping a fair coin and U can identify 
those states that are reached if the coin flip yields head. Then the coin lemma says that 
no matter how the nondeterminism is resolved the probability that either the coin is not 
flipped or the coin is flipped and yields head is at least 1/2. 


Observe that in the definition of the coin event we allow for those executions where no 
coin is flipped. One reason for this choice is to avoid trivial lower bounds due to the fact 
that a generic adversary can always decide not to schedule any transition. Another reason 
is that generally a randomized algorithm is structured so that that if no coin is flipped 
then progress is guaranteed with certainty. Alternatively, a randomized algorithm can be 
structured so that under any valid adversary some coin is flipped. In both cases it is of 
absolute importance to be aware of the existence of executions where no coin is flipped. 
Overlooking those executions is a common source of mistakes. 


2. First occurrence of an action among many. 


In this coin lemma we consider several pairs (a;,U;) of actions and sets of states, and we 
study the probability that either none of the a;’s occur or the action a; that occurs first 
leads to a state of U;. We show that, if for each 7 p; is the lower bound given for (a;, U;) 
by the coin lemma of 1, then the probability mentioned above is at least the minimum of 
the p,’s. 


As an example, consider n processes that run in parallel, and suppose that each process 
can flip a fair coin. Then, the probability that either no process flips a coin or that the 
first process that flips a coin obtains head is at least 1/2. 


3. Lth occurrence of an action among many. 


In this coin lemma we consider the coin event of 2 with the difference that we consider 
the 78 occurrence of an action rather than the first occurrence. The lower bound on the 
probability of this event is the same as the lower bound on the probability of the event 
of 2. 


4. Conjunction of separate coin events. 


In this coin lemma we consider the conjunction of several coin events of the kind of 3. We 
show that if each one of the coin events involves disjoint occurrences of actions, then the 
lower bound on the probability of the conjunction is the product of the lower bounds on 
the probability of each of the involved coin events. 


As an example, consider n processes that run in parallel, and suppose that each process 
can flip a fair coin. For each 7 let x; be either head or tail. Then, the probability that for 
each process 7 either no coin is flipped or the first coin that is flipped yields x; is at least 
1/2". 


Some more general and complex coin lemmas are presented in Section 6.4; several other coin 
lemmas are likely to be derived in the future. Before presenting the simple coin lemmas in full 
detail we give just a rough idea of the coin lemmas of Section 6.4. 


5. Conjunction of separate coin events with multiple outcomes. 
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In this coin lemma we consider again the conjunction of several coin events that involve 
disjoint occurrences of actions. However we allow more freedom. First of all an action is 
paired with more than one set of states, thus allowing the observation of more than one 
outcome; second, we allow for multiple joint observations. 


As an example, the coin lemma says that if n processes run in parallel and each one of 
them can flip a coin, then the probability that at least half of the processes either do not 
flip a coin or flip head is at least 1/2. Similarly, if each process can roll a dice, then the 
probability that if process 1 rolls 1 then the other processes do not roll a number different 
from 1 is at least (1/6)” + 5/6, which is essentially the probability of rolling n dices and 
that either all processes give 1 or process 1 does not give 1. 


6. A generalized coin lemma. 


In this coin lemma we generalize the idea of 5, but this time we do not use actions to 
identify the random draws of interest. The reader is referred to Section 6.4.2 for further 
details. 


6.2.1 First Occurrence of an Action 


Let M be a probabilistic automaton, and let (a,U) be a pair consisting of an action of M and 
a set of states of M. Let FIRST(a,U) be a function that applied to a probabilistic execution 
fragment H of M returns the set of executions a of Qy such that either a does not occur in 
avg, or a occurs in avg! and the state reached after the first occurrence of a is a state of U. 

It is simple to check that FIRST(a,U) is an event schema since, for each probabilistic 
execution fragment H of M, the complement of FIRST(a,U)(H) is the set of executions a of 
Q7y such that action a occurs in avg, and the state reached after the first occurrence of a is 
not a state of U. This set is expressible as a union of cones, and thus it is an event. 

The event schema FIRST(a,U) identifies the first random draw associated with action a 
that occurs in a probabilistic execution fragment H, and requires the outcome of the random 
draw to be in a specific range, namely in U. The intuition behind the use of such a coin event, 
is that a system performs well if the outcome of the first random draw involving a is in U. 
From the definition of FIRST(a,U), we assume also that the system performs well whenever a 
does not occur at all. Thus, if an adversary has the possibility not to schedule a, then it has a 
better chance to degrade the performance of a system by scheduling a. 

The following lemma provides a lower bound to the probability of FIRST(a,U). Informally, 
it states that if whenever there is a transition of M that involves action a the occurrence of a 
implies that a state of U is reached with probability at least p, then p is a lower bound on the 
probability of FIRST(a,U). 


Lemma 6.2.1 Let M be a probabilistic automaton, and let (a,U) be a pair consisting of an 
action of M and a set of states of M. Let p be a real number between 0 and 1 such that for 
each transition (s,P) of M where Pla] > 0, P[U|a] > p. Then, for each probabilistic execution 
fragment H of M, Py|FIRST(a,U)(H)] > p. 


Proof. For convenience denote FIRST(a,U)(H) by FE, and for each state q of H, denote by 
Q(q,U) the set {(a,q’) € QF | Istate(q’) ¢ U}. Let © be the set of states g of H such that 
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action a does not occur in qogé!, and PH al > 0. Then, 


PHE|=>) _ Pa[Cq\Py'((4,7)]. (6.1) 


By expressing PH((a, q')] as a conditional probability and rearranging the expression, we obtain 


PylE] = SO Pa(C PP [a] PFT a)lal | - (6.2) 


qEO (a, EQ(g,U) 


From the definition of a probabilistic execution fragment and the definition of Q(¢, U), for each 
element g of © there is a combined transition tr = 5°; p;tr; of M such that tr? = qtr and 


_ Pir(U Na) So; pPirfU aa 
PP a, @)la] = PiU la] = SS = SS . (6.3) 
— oN? . Pip pj Pry, 
(aghenteD) ir [a di PiPer, lal 
By multiplying and dividing each it* summand of the enumerator by P,,,[a], using the hypoth- 
esis of the lemma, i.e., for each i P;,,[U Na] < (1—>p), and simplifying algebraically, from (6.3) 
we obtain 


PR [(a, gla] < = p). (6.4) 


(a,q’)EQ(g,U) 


By using (6.4) in (6.2) we obtain 


Py[E] < Ap) [x nutcar) (6.5) 


qEO 


Furthermore, the subexpression >?,¢o Py[C\ PF [a] is the probability that @ occurs in H, which 
is at most 1. Thus, 


Py(B] < (1p). (6.6) 


This completes the proof. | 


6.2.2 First Occurrence of an Action among Many 


The event schema FIRST(a,U) can be generalized to account for the first action that occurs 
among several possible ones. Let M be a probabilistic automaton, and let (a1, U1),...,(@n, Un) 
be pairs consisting of an action of M and a set of states of M such that the actions a; are 
all distinct. Then define FIRST((a1,U1),...,(@n,Un)) to be the function that applied to a 
probabilistic execution fragment H of M returns the set of executions a of Oy such that either 
none of the a;’s occurs in avg, or some of the a;’s occur in age, and if a; is the first of those 
actions that occurs, then the state reached after the first occurrence of a; is a state of U;. 

It is simple again to check that FIRST((a1,U1),...,(@n,U;,)) is an event schema since, for 
each probabilistic execution fragment H, the complement of FIRST ((a1,U1),...,(@n,Un))(H) 
can be expressed as a union of cones. 

Lemma 6.2.1 extends to this case. 
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Lemma 6.2.2 Let M be a probabilistic automaton, and let (a,,U1),...,(a@n, Un) be pairs con- 
sisting of an action of M and a set of states of M such that the actions a; are all distinct. Let 
{pi}izi,...n be a collection of real numbers between 0 and 1 such that for each i, 1 <i< n, 
and each transition (s,P) of M where Pla;] > 0, P[U|a;] > p;. Then, for each probabilistic 
execution fragment H of M, Py[FIRST((a1,U1),...,(@n,Un))(H)] > min(pi,..-,pn)- 


Proof. Let V be {a1,...,a,}, and let p be the minimum of {p1,...,p,}. For convenience, 
denote FIRST((a1,U1),.--,(dn,Un))(H1) by FE, and for each state q of H, denote by Q(q, EF) 
the set Useta, jn} (Gis 7’) € QF | istate(q’) € U;}. Then, for each transition (4, PH) of H such 
that P#[V] > 0, 


PHQ(q, E)|V] < (1 = p). (6.7) 


q 

To prove (6.7), let, for each i = 1,...,n, Q(g,a;, U;) denote the set {(a;,q') € QF | istate(q’) ¢ 

U;}. Then, 

PEIQG, EV) = Dd) PP (2, 4:,05)|V).- (6.8) 
7€{1,....n} 

By using conditional probabilities, Equation (6.8) can be rewritten into 

PEIQQ EV] = So PP lal VIPS(QG, ai, Oi)la). (6.9) 

7€{1,....n} 


Following the same argument as in the proof of Lemma 6.2.1, for each 2, PHO, a;,U;)|ai] < 
(1 — p); moreover, >; P#[a;|V] = 1. Thus, (6.7) follows directly. 

The rest of the proof follows te lines of the proof of Lemma 6.2.1. Let © be the set of states 
q of H such that no action of V occurs in qogé!, and PHI] > 0. Then, 


PHE|=>) _ Pal CP? (4,9): (6.10) 


PulE] = SO Pal P71] » _ PH((a,q)IV) | - (6.11) 


The subexpression D(a q’)EQ(9,E) 


(1 — p) from (6.7). Thus, 


PH (a, Q)|V] is P?[Q(¢, £)|V], which is less than or equal to 


P(E] < (1p) [x Pu{C,\P# "] | (6.12) 


Furthermore, the subexpression }°,¢o Py[C PE [V] is the probability that an action from V 
occurs in A, which is at most 1. Thus, 


Py[E] < (1—p). (6.13) 


This completes the proof. | 
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6.2.3 I-th Occurrence of an Action among Many 


In the definition of FIRST we have considered the first action among a given set that occurs 
in a probabilistic execution fragment H. However, the results for FIRST are valid also if 
we consider the i** occurrence of an action instead of the first occurrence. This observation 
suggests a new more general event schema. 

Let M be a probabilistic automaton, and let (a1, U1),...,(a@n,Un) be pairs consisting of 
an action of M and a set of states of M such that the actions a; are all distinct. Then 
define OCC (i, (a1, U1),...,(@n, Un)) to be the function that applied to a probabilistic execution 
fragment H of M returns the set of executions a of Oy such that either there are less than 7 
occurrences of actions from {a1,...,@,} in avg’, or there are at least 7 occurrences of actions 
from {a1,...,@,}, and, if a; is the action that occurs as the i** one, then the state reached 
after its occurrence is a state of U;. 

Since in the proof of Lemma 6.2.2 we never use the fact that it is the first occurrence of an 
action that is considered, Lemma 6.2.2 carries over to the it® occurrence trivially. 


Lemma 6.2.3 Let M be a probabilistic automaton, and let (a,,U1),...,(an, Un) be pairs con- 
sisting of an action of M and a set of states of M such that the actions a; are all distinct. Let 
{pj }j=t,..4n be a collection of real numbers between 0 and 1 such that for each j € {1,...,n} 
and each transition (s,P) of M where Pla;| > 0, P[U|a;] > p;. Then, for each probabilistic 
execution fragment H of M, Py[OCC(i, (a1, U1),..-, (an, Un))(H)| > min(pi,..., pn). a 


6.2.4 Conjunction of Separate Coin Events 


In this section we study what happens if we consider several events of the kind OCC together. 
In order to simplify the notation, we consider only event schemas of the kind OCC(i,(a,U)) 
since, as we have seen in the proof of Lemma 6.2.2, the case with multiple actions can be 
reduced to the case with a single action. 

The lemma that we prove states that if we consider several separate coin events, i.e., coin 
events that involve different random draws, each one with its own lower bound, then the lower 
bound of their conjunction is the product of the lower bounds. In other words, an adversary 
can introduce dependencies by increasing the probability of the conjunction of events, but it 
can never decrease the probability below the value that we would get by considering all the 
events to be independent. 


Lemma 6.2.4 Let M be a probabilistic automaton, and let (ky,a1,U1),...,(kn,@n,Un) be a 
collection of triplets consisting of a natural number, an action of M and a set of states of 
M, such that the pairs (k;,a;) are all distinct. Let {pj}j=1,...n be a collection of real num- 
bers between 0 and 1 such that for each j € {1,...,n} and each transition (s,P) of M 
where Pla;] > 0, P[U|a;] > p;. Then, for each probabilistic execution fragment H of M, 
Py[OCC(hr, (a1, U1))C) ++ OCC (Ra, (dn, Un))(A)] > pie Pn. 


Proof. For each J C {1,...,n}, denote a generic event schema NjerOCC(ki, (ai, U;)) by er. 
For each i = 1,...,n and each state g of H, denote by Q(q,7%,U;) the set {(aj,q/) € QF | 
Istate(q’) € U;} of pairs where a; occurs and U; is reached, and denote by Q(q,i,U;) the set 
{(ai,q) € QI | Istate(q’) ¢ Uj} of pairs where a; occurs and U; is not reached. For each action 
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a and each state q of H, let a(q) denote the number of occurrences of action a in qogs!. For 
each 7 = 1,...,n, let O; be the set of states g of H such that each action a;,1 <j <n occurs 
less than fk; times in qogs, action a; occurs k; — 1 times in gogé!, and PH ai] > 0. For each 
i= 1,...,n and each state q of H such that a;(q) < kj, let OCC(ki, (a;, U;))>q denote the event 
schema OCC(k; — a;(q), (ai, U;)). Finally, for each J C {1,...,n} and each suitable state q of 
H, let eyeg denote the event schema NjerOCC( ki, (a;, Ui) og. 

We prove the lemma by induction on n. If n = 1, then the result follows directly from 
Lemma 6.2.1. Otherwise, 


Puflét..nE= D> Dd) ParlCy] Paid] 


7E{1,...,.n} qeO; (a4,q')EQ(q,2,0%) 


+ ( S- PH (ai, 11 Pre er TTT) . (6.14) 
(a;,9') 


€2(g,1,0%) 


The first summand of Expression (6.14) expresses the probability that action a; occurs from q¢ 
and leads to a state not in U;; the second summand expresses the probability that a; occurs, leads 
to a state of U;, and from the reached state something happen so that the resulting execution 
is not in e),...,(H). From induction, and by using conditional probabilities, we obtain 


Pyfét,.n(E)< SS SD Pa[Cy] PE fai] YS PF (ai. d)\ai] 


7€{1,....n} 7E0; (aig )EQ(q,i,7%) 


+ ( » PEP P)) ; (6.15) 
(ai.9’) 


€2(g,1,0%) 


Let, for each 7 and each q, pig = PH [Q(q, t, U;)|ai]. Then, (6.15) becomes 


Pulet,...nZ)] 


< SO YS PalC PH [ail = pig) + 1 = pi ++ pi-apita +++ Pn)Pig)s (6.16) 
tE{1,....n} ¢EOs 


which becomes 
Prlet,.n ED) < SS Pa [Cy PP fail. — pi ++ pi-1PigPigt +++ Pn) (6.17) 
tE{1,....n} qEO; 


after simple algebraic simplifications. Using the same argument as in the proof of Lemma 6.2.1, 
for each 7 and each q, pig > pi. Thus, 


Palen) <S S> YS PulC,)PM al — pr pn). (6.18) 
tE{1,....n} qEO; 


Finally, observe that )7je41,....n} 2.qe0; Py{Cq] PF [ai] is the probability that for some 7 action 
a; occurs at least k; times. Thus, 


Pulet,..jn(H)] < (1 — pis +pn)- (6.19) 


This completes the proof. | 
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Figure 6-1: The Dining Philosopher problem with 6 philosophers. 


6.3. Example: Randomized Dining Philosophers 


In this section we apply the methodology presented so far to prove the correctness of the Ran- 
domized Dining Philosophers algorithm of Lehmann and Rabin [LR81]. The proof is structured 
in two levels. The high level proof consists of a collection of progress statements that are con- 
catenated together; the low level proof consists of the proofs of the statements of the high level 
proof. The low level proof is based on the coin lemmas. 


6.3.1 The Problem 


There are n philosophers sat at a round table. Each philosopher has a plate in from of him, a 
fork on its left, and a fork on its right. The left fork is shared with his left neighbor philosopher, 
and the right fork is shared with his right neighbor philosopher. At the center of the table there 
is a bowl full of spaghetti. Figure 6-1 illustrates the situation for n = 6. Each philosopher 
goes repeatedly through phases where he is thinking and where he is eating. However, each 
philosopher needs both of its forks in order to eat. The problem is the following: 


“What procedure should each philosopher follow to get his forks and to put them 
down in order to make sure that every philosopher that is hungry will eventually be 
able to eat?” 


A simpler problem is the following. 


“What procedure should each philosopher follow to get his forks and to put them down 
in order to make sure that whenever somebody is hungry somebody will eventually 
be able to eat?” 


The second problem is simpler than the first problem since it allows for some philosopher 
to starve. It is known from [LR81] that there is no symmetric solution even for the simple 
dining philosophers problem, i.e., there is no deterministic solution for the dining philosophers 
problem where each philosopher follows exactly the same protocol; some mechanism to break 
the symmetry is necessary. In the algorithm of Lehmann and Rabin each philosopher follows 
exactly the same protocol and randomness is used to break the symmetry. 
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Shared variables: Res; € {free,taken}, j = 1,...,n, initially free. 
Local variables: u; € {left,right}, i=1,...,n 


Code for process ?: 


0. try ** beginning of Trying Section ** 
1. < uj — random> ** choose left or right with equal probability ** 
2. < if Res;;,,,) = free then 
Res(;.,;) ‘= taken ** pick up first resource ** 
else goto 2. > 
3. < if Res; opp(u;)) = free then 
Resi, opp(uj)) = taken; ** pick up second resource ** 
goto 5. > 
4, < Res¢;y,) '= free; goto 1.> ** nut down first resource ** 
5. crit ** end of Trying Section ** 
** Critical Section ** 
6. exit ** beginning of Exit Section ** 
7. <u; — left or right ** nondeterministic choice ** 
Res(j,opp(u;)) = Free > ** put down first resources ** 
8. < Res(;,,) := free > ** put down second resources ** 
9. rem ** end of Exit Section ** 


** Remainder Section ** 


Figure 6-2: The Lehmann-Rabin algorithm. The operations between angular brackets are 
performed atomically. 


6.3.2 The Algorithm 


Each hungry philosopher proceeds according to the following protocol. 


1. Flip a fair coin to choose between the left and the right fork. 
. Wait for the chosen fork to become free and get it. 
3. Try to get the second fork: 


if it is free, then get it; 
if it is taken, then put down the first fork and go to 1. 
4, Eat. 


Each philosopher that has terminated to eat puts down his forks one at a time. The intuition 
behind the use of randomness is that the actual protocol used by each philosopher is determined 
by an infinite sequence of random coin flips. Thus, with probability 1 each philosopher follows 
a different protocol. 

Figure 6-2 gives a more precise representation of the protocol, using a terminology that 
is closer to computer science; thus, a philosopher is called a process, and a fork is called a 
resource. A philosopher who is thinking is said to be in its reminder region; a philosopher 
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Figure 6-3: Numbering processes and resources in the Dining Philosophers problem. 


who is eating is said to be in its critical region; a philosopher who is trying to get its forks is 
said to be in its trying region; and a philosopher who is putting down its forks is said to be in 
its exit region. The n resources (forks) are represented by n shared variables Res,..., Res, 
each of which can assume values in {free,taken}. Each process (philosopher) 7 ignores its 
own name and the names of its adjacent resources. However, each process 7 is able to refer 
to its adjacent resources by relative names: Resi; j¢¢¢) is the resource located to the left, and 
Res(; rignt) is the resource to the right of 2. Each process 7 has a private variable u;, whose value 
is in {left,right}, which is used either to keep track of the resource that process i currently 
holds, or, if no resource is held, to keep track of the resource that process 2 is going to take 
next. For notational convenience we define an operator opp that complements the value of its 
argument, i.e., opp(right) = left and opp(left) = right. 

We now define a probabilistic automaton M that represents the evolution of n philosophers. 
We assume that process 2+ 1 is on the right of process 7 and that resource Res; is between 
processes i and i+ 1 (see Figure 6-3). We also identify labels modulo n so that, for instance, 
process n + 1 coincides with process 1. 

A state s of M is a tuple (Xj,..., Xn, Res1,..., Res, ) containing the local state X; of each 
process 7, and the value of each resource Res;. Each local state X; is a pair (pc;, u;) consisting 
of a program counter pe; and the local variable u;. The program counter of each process keeps 
track of the current instruction in the code of Figure 6-2. Rather than representing the value 
of the program counter with a number, we use a more suggestive notation which is explained 
in Table 6.1. Also, the execution of each instruction is represented by an action. Actions try,, 
crit;, rem;, exit; are external; all the other actions are internal. 

The start state of M assigns the value free to all the shared variables Res;, the value R to 
each program counter pc;, and an arbitrary value to each variable u;. The transition relation 
of M is derived directly from Figure 6-2. For example, for each state where pc; = F there is 
an internal transition labeled with flip, that changes pce; into W and assigns left to u; with 
probability 1/2 and right to u; with probability 1/2; from each state where X; = (W,left) 
there is a transition labeled with wait; that does not change the state if Res(;,e¢) = taken, 
and changes pe; into S and Resi; 1e¢¢) into taken if Res(; jer) = free; for each state where 
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Nr. pe; Action Informal meaning 

0 R try; Reminder region 

1 I flip, Ready to Flip 

2 W wait; Waiting for first resource 

3 $ second; Checking for Second resource 
4 D drop; Dropping first resource 

5 P crit; Pre-critical region 

6 C exit; Critical region 

7 Ey dropf; Exit: drop First resource 

8 Es drops; Exit: drop Second resource 
9 ER rem; Exit: move to Reminder region 


Table 6.1: Program counter and action names for the Lehmann-Rabin algorithm. 


pc; = Ef there are two transitions labeled with action dropf;: one transition sets u; to right 
and makes Res; j.¢¢) free, and the other transition sets u; to left makes Res(; pignty free. The 
two separate transitions correspond to a nondeterministic choice that is left to the adversary. 

The value of each pair X; can be represented concisely by the value of pc; and an arrow 
(to the left or to the right) which describes the value of u;. Thus, informally, a process 7 is in 
state S or D (resp. 5 or D) when 7 is in state S$ or D while holding its right (resp. left) 
resource; process 7 is in state W (resp. W) when 7 is waiting for its right (resp. left) resource 
to become free; process 7 is in state Es (resp. Es) when ? is in its exit region and it is still 
holding its right (resp. left) resource. Sometimes we are interested in sets of pairs; for example, 
whenever pc; = F the value of u; is irrelevant. With the simple value of pc; we denote the set of 
the two pairs {(pc;, left), (pce;,right)}. Finally, with the symbol # we denote any pair where 
pe; © {W, 5S, D}. The arrow notation is used as before. 

For each state s = (X4,..., Xn, Resi,...,Res,,) of M we denote X; by X;(s) and Res; by 
Res;(s). Also, for any set St of states of a process i, we denote by X; € St, or alternatively 
X; = St the set of states s of M such that X;(s) € St. Sometimes we abuse notation in the 
sense that we write expressions like X; ¢ {F,D} with the meaning X; € FU D. Finally, we 
write X; = EF for X; = {Er, Es, Ep}, and we write X; = T for X; € {F,W,S,D,P}. 


6.3.3. The High Level Proof 


In this section we give the high level proof that the algorithm of Lehmann and Rabin guarantees 
progress, i.e., that from every state where some process is in its trying region, some process 
enters eventually its critical region with probability 1. We assume that each process that is 
ready to perform a transition is allowed eventually to do so: process 2 is ready to perform a 
transition whenever it enables an action different from try; or exit;. Actions try; and exit; 
are under the control of the user (a philosopher decides whether to eat or think), and hence, 
by assumption, under the control of the adversary. 

Formally, consider the probabilistic automaton M of Section 6.3.2. Define an extended 
execution a of M to be fair iff for each process 7 either a is finite and its last state enables 
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try, or exit;, or a is infinite and either actions of process 2 occur infinitely many times in a 
or @ = a; ~ a2 and all the states of a2 enable either try; or exit;. Define Fairadvs to be the 
set of adversaries A for M such that, for every finite execution fragment a of M the elements 
of Qorevee(M,A,a) are extended fair execution fragments of M. Then Fairadvs is finite-history- 
insensitive: if A is an adversary of Fatradvs and q is a finite execution fragment of MW, then it 
is easy to verify that the adversary A, such that 


_ J Afarg) ifg<a 
Aq() = A(a) — otherwise 


is an adversary of Fairadvs. Let rstates(M) denote the set of reachable states of M. Let 
T = {s € rstates(M) | 4;X;(s) € {T}} 


denote the sets of reachable states of M where some process is in its trying region, and let 


C = {s € rstates(M) | 4;X;(s) = C} 


denote the sets of reachable states of 44 where some process is in its critical region. We first 
show that 
T qjahuredes C, (6.20) 
i.e., that, starting from any reachable state where some process is in its trying region, for all 
the adversaries of Fairadvs, some process enters its critical region eventually with probability at 
least 1/8. Note that (6.20) is satisfied trivially if some process is initially in its critical region. 
Our proof is divided into several phases, each one concerned with the property of making 
some partial progress toward C. The sets of states associated with the different phases are 


expressed in terms of 7,RT,F,G,P, and C. Here, 
RT © {s€T|V:Xi(s) €{Er, B.T}} 


is the set of states where at least one process is in its trying region and where no process is in 
its critical region or holds resources while being in its exit region. 


F = {s€RT | 4;X;(s) = F} 


is the set of states of RT where some process is ready to flip a coin. 


P = {s €rstates(M) | 3;X;(s) = P} 


is the sets of reachable states of M where some process is in its pre-critical region, i.e., where 
some process is ready to enter its critical region. The set G is the most important for the 
analysis. To motivate the definition, we define the following notions. We say that a process i 
is committed if X; € {W,S}, and that a process i potentially controls Res; (resp. Res;_1) if 
X;€ {W, 5, D} (resp. X; € {W, S, D}). Informally said, a state in RT is in G if and only 
if there is a committed process whose second resource is not potentially controlled by another 
process. Such a process is called a good process. Formally, 


G = {sERT|I; 
X;j(s) € {W, Ss} and Xi41(s) € {Ep, R, F, #}, or 
Xi(s) € {W, S$} and X;-1(s) € {Er, RB, F, #}} 
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Reaching a state of G is a substantial progress toward reaching a state of C. Somehow, a good 
state is a place where the symmetry is broken. The progress statements of the proof are the 


following. 
T > RT UC (Proposition 6.3.3), 
RT > FU GUP_ (Proposition 6.3.16), 
F arr GUP (Proposition 6.3.15), 
G ia P (Proposition 6.3.12), 
P—-C (Proposition 6.3.1). 


The first statement says that eventually every process in its exit region relinquishes its resources. 
In this way we avoid to deal with resources held by processes who do not want to enter the 
critical region. The second statement says that eventually either a good state is reached, or a 
place where some process is ready to flip its coin is reached. The flipping points are potential 
points where the symmetry is broken, and thus reaching a flipping point means progress. The 
third statement says that from a flipping point there is probability 1/2 to reach a good state. 
Finally, the fourth statement says that from a good state there is probability 1/4 to be ready 
to enter the critical region. By combining the statements above by means of Proposition 5.5.3 
and Theorem 5.5.2 we obtain 

T ie C, (6.21) 
which is the property that was to be proven. Observe that once some process is in the trying 
region there is always some process in the trying region until some process reaches the critical 
region. Formally, M satisfies T UnlessC. Thus, Proposition 5.5.6 applies, leading to 


T—C. (6.22) 


6.3.4 The Low Level Proof 


In this section we prove the five progress statements used in Section 6.3.3. The proofs are 
detailed operational arguments. The main point to observe is that randomness is handled 
exclusively by the coin lemmas, and thus, any technique for the verification of ordinary automata 
could be applied as well. 

For the sake of clarity, we do not prove the relations in the order they were presented. 
Throughout the proof we abuse notation by writing expressions of the kind FIRS7T(flip,, left) 
for the event schema FIRST(f1ip;,, {s € states(M) | X;(s) = W}). We write also sentences of 
the form “If FIRST(flip,;,left) then ¢” meaning that for each valid probabilistic execution 
fragment H, each element of FIRST(flip,,left)(/) satisfies ¢. 


Proposition 6.3.1 If some process is in P, then some process enters C, 1.€., 


Proof. Let 7 be the process in P. Then, from the definition of Fairadvs, process 7 is scheduled 
eventually, and enters C. | 
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Lemma 6.3.2 If some process is in its Exit region, then it will eventually enter R. 


Proof. The process needs to perform two transitions to relinquish its two resources, and then 
one transition to send a rem message to the user. Every adversary of Fairadvs guarantees that 
those three transitions are performed eventually. | 


Proposition 6.3.3 7 — RT UC. 


Proof. From Lemma 6.3.2, every process that begins in Pp or Fs relinquishes its resources. 
If no process begins in C’ or enters C in the meantime, then the state reached at this point is 
a state of RT; otherwise, the starting state or the state reached when the first process enters 
C' is a state of C. | 


We now turn to the proof of G Va P. The following lemmas form a detailed cases analysis 


of the different situations that can arise in states of G. Informally, each lemma shows that a 
specific coin event is a sub-event of the properties of reaching some other state. A preliminary 
lemma is an invariant of MM, which guarantees that the resources are held by those processes 
who think to be holding them. 


Lemma 6.3.4 For each reachable state s of M and each 1, 1 < i < n, Res; = taken iff 
Xi(s)€{5,D,P,C, Ep, Es} or Xi41(s) €{5,D,P,C, Ep, Es}. Moreover, for each reachable 
state s of M and each i, 1 <i <n, it is not the case that X;(s) € {5,D,P, C, Ep, Es} and 
Xigi(s) € {55 Dd, PC, Ep, Es}, i.e., only one process at a time can hold one resource. a 


Proof. The proof of this lemma is a standard proof of invariants. Simply verify that the two 
properties are true for the start states of Mf and are preserved by each transition of M. | 


Lemma 6.3.5 


1. Let X;-, € {Ep, R, F} and X; = W. If FIRST (flip,_,,left), then, eventually, either 
Xj-1 = Poor X;=5S. 


2. Let X;-1 = D and X;= W. If FIRST(f£lip;_,, left), then, eventually, either X;_1 = P 
or X; = S. 


3. Let X;-1 = S and X; = W. If FIRST(#lip,_,,left), then, eventually, either X;1 = P 
or X; = S. 


4. Let X;-1 = W and X; = W. If FIRST(flip,;_,,left), then, eventually, either X;1 = P 
or X; = S. 


Proof. The four proofs start in the same way. Let s be a state of M satisfying the respective 
properties of items / or 2 or 3 or 4. Let A be an adversary of Fairadvs, and let a be an 
execution Of Qorerec(M,{s},A) Where the result of the first coin flip of process ¢ — 1, if it occurs, 
is left. 
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1. By hypothesis and Lemma 6.3.4, 2 — 1 does not hold any resource at the beginning of a 
and has to obtain Res;_2 (its left resource) before pursuing Res;_1. From the definition 
of Fairadvs, i performs a transition eventually in a. If 2— 1 does not hold Res;_; when 
t performs this transition, then ¢ progresses into configuration S$. If not, it must be the 
case that i— 1 succeeded in getting it in the meanwhile. But, in this case, since 2 — 1 flips 
left, Res;_1 was the second resource needed by 7 — 1 and i — 1 therefore entered P. 


2. If X; = S eventually, then we are done. Otherwise, process i — 1 performs a transition 
eventually. Let a = a,~ a2 such that the last transition of a, is the first transition taken 
by process 7— 1. Then X;_1(fstate(az)) = F and X;(fstate(a2)) = W. Since process 
t—1 did not flip any coin during a, from the finite-history-insensitivity of Fairadvs and 
Item / we conclude. 


3. If X; = S eventually, then we are done. Otherwise, process i — 1 performs a transition 
eventually. Let a = a,~ a2 such that the last transition of a, is the first transition taken 
by process i — 1. If X;_1(fstate(az)) = P then we are also done. Otherwise it must be 
the case that X;_1(fstate(a2)) = D and X;(fstate(a2)) = W. Since process i — 1 did not 


flip any coin during a,, from the finite-history-insensitivity of Fatradvs and Item 2 we 
conclude. 


4. If X; = S eventually, then we are done. Otherwise, process 2 checks its left resource 
eventually and fails, process 7 — 1 gets its right resource before, and hence reaches at 
least state S. Let @ = a1 ~ ag where the last transition of a1 is the first transition of a 
that leads process 7 — 1 to state 5. Then X;_1(fstate(a2)) = $ and X;(fstate(az)) = W. 
Since process i — 1 did not flip any coin during a1, from the finite-history-insensitivity of 
Fairadvs and Item 3 we conclude. | 


Lemma 6.3.6 Assume that X;_; € {Ep,R,T} and X; = W. If FIRST(£lip,_,,left), then, 
eventually, either X;_1 = P or X;=S. 


Proof. Follows directly from Lemma 6.3.5 after observing that X;_1 € {Ep, R,T} is equivalent 
to X;_1 € {Ep, R, F,W,S,D, P}. a 


The next lemma is a useful tool for the proofs of Lemmas 6.3.8, 6.3.9, and 6.3.10. 


Lemma 6.3.7 Let X; € {W, 5} or X; © {Ep, R, F, D} with FIRST(flip;,left). Further- 
more, let Xi41 € {W, S} or Xiqi € {Epn, RF, D} with FIRST(flip;,,,right). Then the 
first of the two processes i or i+ 1 testing its second resource enters P after having performed 
this test (if this time ever comes). 


Proof. By Lemma 6.3.4 Res; is free. Moreover, Res; is the second resource needed by both i 
and i+ 1. Whichever tests for it first gets it and enters P. | 


Lemma 6.3.8 Jf X; = Ss and Xj44 € {W, Sh then, eventually, one of the two processes i or 
i+1 enters P. The same result holds if X; € {W, Ss} and Xj4, = S. 
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Proof. Being in state S, process i tests its second resource eventually. An application of 
Lemma 6.3.7 finishes the proof. | 


Lemma 6.3.9 Let X; = Ss and Xi41 € {Er, R, F, Dj. If FIRST (flip,,,, right), then, even- 
tually, one of the two processes i or it+1 enters P. The same result holds if X; € {Ep, R, F, D}, 
X41 = S and FIRST(f1ip,, left). 


Proof. Being in state S, process i tests its second resource eventually. An application of 
Lemma 6.3.7 finishes the proof. | 


Lemma 6.3.10 Assume that X;-1 € {Ep, R,T}, X; = W, and Xj41 € {Ep, R, F, W, D}. Lf 
FIRST(flip,;_,,left) and FIRST(flip;,,,right), then eventually one of the three processes 
t—1,t ort4+1 enters P. 


Proof. Let s be a state of M such that X;1(s) € {Er,R,T}, Xi(s) = W, and Xi41(s) € 
{Ep, R, F, Ww, D}. Let A be an adversary of Fazradvs, and let a be an extended execution of 
Qorerec(M,{s},A) Where the result of the first coin flip of process i — 1 is left and the result 
of the first coin flip of process +1 is right. By Lemma 6.3.6, eventually either process 
t — 1 reaches configuration P in a or process 7 reaches configuration Ss in a. If t—1 reaches 
configuration P, then we are done. If not, then let a = a; ~ ag such that Istate(a,) is the 
first state s’ of a with X;(s') = S. Ifi+1 enters P before the end of a1, then we are done. 
Otherwise, X;41(fstate(a2)) is either in {W, 5 } or it is in {Bp, R,F, D} and process 7 + 1 
has not flipped any coin yet in a. From the finite-history-insensitivity of Fairadvs we can then 
apply Lemma 6.3.7: eventually process 7 tests its second resource and by Lemma 6.3.7 process 
i enters P if process i+ 1 did not check its second resource in the meantime. If process 7 + 1 


checks its second resource before process 2 does the same, then by Lemma 6.3.7 process 7+ 1 
enters P. | 


Lemma 6.3.11 Assume that Xi4o € {Fp, R,T}, Xian = W, and X; € {Ep, R,F, W, D}. If 
FIRST(flip,;,left) and FIRST(flip;,.,right), then eventually one of the three processes i, 
t+1 ori4+ 2, enters P. 


Proof. The proof is analogous to the one of Lemma 6.3.10. This lemma is the symmetric case 
of Lemma 6.3.10. | 


Proposition 6.3.12 Starting from a global configuration in G, then, with probability at least 
1/4, some process enters P eventually. Equivalently: 


G it P. 
Proof. Lemmas 6.3.8 and 6.3.9 jointly treat the case where X; = s and Xi41 € {Er, R, F, #} 
and the symmetric case where X; € {Ep, R, F, i} and X44, = 53 Lemmas 6.3.10 and 6.3.11 
jointly treat the case where X; = Ww and Xj41 € {Er R,F, W,D} and the symmetric case 
where X; € {Ep, R,F, W, D} and Xj41 = W 


= . 
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Specifically, each lemma shows that a compound event of the kind FIRST (flip,,x) and 
FIRST(flip,,y) leads to P. Each of the basic events FIRST(flip;, x) has probability at least 
1/2. From Lemma 6.2.4 each of the compound events has probability at least 1/4. Thus the 
probability of reaching P eventually is at least 1/4. | 


We now turn to F Te GUP. The proof is divided in two parts and constitute the global 


argument of the proof of progress, i.e., the argument that focuses on the whole system rather 
than on a couple of processes. 


Lemma 6.3.13 Start with a state s of F. If there exists a process i for which X;(s) = F and 
(Xj-1, Xigi) 4 (i, #), then, with probability at least 1/2 a state of GUP is reached eventually. 


Proof. If s € GUP, then the result is trivial. Let s be a state of F —(GUP) and let i be such 
that X;(s) = F and (Xj-1, Xi4i1) F (i, if). Assume without loss of generality that X;41 4 #, 
ie., Nig, € {Ep, R, F, i}. The case for X;1 4 # is similar. Furthermore, we can assume 
that Xi41 € {Er, R, F, D} since if X;41 € {W, S} then s is already in G. We show that the 
event schema FIRST((flip;,left),(flip;,,,right)), which by Lemma 6.2.2 has probability 
at least 1/2, leads eventually to a state of GU P. Let A be an adversary of Fairadvs, and let 
a be an extended execution of 2 pcxee(M,{s},A) Where if process ? flips before process i + 1 then 
process ¢ flips left, and if process i+ 1 flips before process i then process 7 + 1 flips right. 
Then, eventually, 7 performs one transition and reaches W. Let j € {7,i+ 1} be the first of 
tandi+1 that reaches W and let s, be the state reached after the first time process j reaches 
W. If some process reached P in the meantime, then we are done. Otherwise there are two 
cases to consider. If 7 = 7, then, flip; yields left and X;(s1) = W whereas X;41 is (still) in 
{Er, R, Fy D}. Therefore, s; € G. If j =i+1, then flip;,, yields right and Xj41(s1) = Ww 
whereas X;(s1) is (still) F. Therefore, s1 € G. = 


Lemma 6.3.14 Start with a state s of F. If there exists a process i for which X;(s) = F and 
(Xj-1(s), Xiqa(s)) = (i, i). Then, with probability at least 1/2, a state of GU P is reached 
eventually. 


Proof. The hypothesis can be summarized into the form (X;_-1(s), Xi(s), Xi¢i(s)) = (#, F, #). 
Since 2—1 and z+1 point in different directions, by moving to the right of ++1 there is a process 
k pointing to the left such that process & + 1 either points to the right or is in {Fp, R, F, P}, 
ie., Xz(s) € {W, 5, D} and X,41(s) € {Fr, RF, W,S,D,P}. 

If Xn(s) € {W, 5} and Xy41(s) # P then s € G and we are done; if Xp41(s) = P then 
s € P and we are done. Thus, we can restrict our attention to the case where X;(s) = D. 

We show that FIRST((flip,, left), (flip,,,,right)), which by Lemma 6.2.2 has proba- 
bility at least 1/2, leads eventually to GUP. Let A be an adversary of Fairadvs, and let a 
be an extended execution of Qprerec(M,{s},A) Where if process k flips before process k + 1 then 
process & flips left, and if process k + 1 flips before process & then process & + 1 flips right. 

Then, eventually, process k performs at least two transitions and hence goes to configuration 
W. Let j € {k,k+1} be the first of k and &+1 that reaches W and let s; be the state reached 


after the first time process 7 reaches W. If some process reached P in the meantime, then we are 
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done. Otherwise, we distinguish two cases. If 7 = k, then, flip, yields left and X;,(s1) = Ww 
whereas X;,41 is (still) in {Fp, R, F, #}. Therefore, s; € G. If 7 =k +1, then flip,,, yields 
right and X;41(s1) = W whereas X;(s1) is (still) in {D, FP}. Therefore, 51 € G. = 


Proposition 6.3.15 Start with a state s of F. Then, with probability at least 1/2, a state of 
G UP is reached eventually. Equivalently: 


F—-+GUP. 
1/2 
Proof. The hypothesis of Lemmas 6.3.13 and 6.3.14 form a partition of F. | 


Finally, we prove RT > FUGUP. 


Proposition 6.3.16 Starting from a state s of RT, then a state of F UGUP is reached 
eventually. Equivalently: 


RT —> FUGUP. 


Proof. Let s be a state of RT. If s ¢ FUGUP, then we are trivially done. Suppose 
that s ¢ FUGUP. Then in s each process is in {Fp, R,W,$,D} and there exists at least 
process in {W, 5, D}. Let A be an adversary of Fairadvs, and let a be an extended execution 
Of Qoreree(M{s},A)- 

We first argue that eventually some process reaches a state of {5, D, F} in a. This is trivially 
true if in state s there is some process in {.$, D}. If this is not the case, then all processes are 
either in Ep or R or W. Eventually, some process in R or W performs a transition. If the 
first process not in Fp performing a transition started in Kp or R, then it reaches F and we 
are done; if the first process performing a transition is in W, then it reaches $ since in s no 
resource is held. Once a process ? is in {.5, D, F}, then eventually process 7 reaches either state 
F or P, and we are done. | 


6.4 General Coin Lemmas 


The coin lemmas of Section 6.2 are sufficiently general to prove the correctness of the Random- 
ized Dining Philosophers algorithm of Lehmann and Rabin. However, there are several other 
coin events that are relevant for the analysis of distributed algorithms. For example, the toy 
resource allocation protocol that we used in Chapter 5 cannot be verified yet. In this section 
we present two general coin lemmas: the first one deals with multiple outcomes in a random 
draw; the second one gives a generalization of all the coin lemmas presented in the thesis. 
Unfortunately, generality and simplicity are usually incompatible: the two coin lemmas of this 
section are conceptually more complicated than those of Section 6.2. 


6.4.1 Conjunction of Separate Coin Events with Multiple Outcomes 


The coin lemma of Section 6.2.4 deals with the result of the intersection of several coin events. 
Thus, for example, if each coin event expresses the process of flipping a coin, then the coin 
lemma of Section 6.2.4 can be used to study the probability that all the coins yield head. 
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However, we may be interested in the probability that at least half of the coins yield head, 
or in the probability that exactly 5 coins yield head. The coin lemmas of Section 6.2 are not 
adequate. Suppose now that we use each coin event to express the process of rolling a dice. 
The coin events of Section 6.2 are not adequate again since they can deal only with binary 
outcomes: we can observe only whether a specific set U is reached or not. How can we express 
the event that for each number 2 between | and 6 there is at least one dice that rolls 7? 

In this section we define a coin event and prove a coin lemma that can deal with the scenarios 
outlined above. Let M be a probabilistic automaton, and let S be a set of n tuples {1,..., an}, 
where for each 7, 1 <i <n, a; is a tuple (a;,U;1,..., Ui.) consisting of an action of M and k 
pairwise disjoint sets of states of M7. Let the actions a; be all distinct. Let K be a set of tuples 
((1,91),---,(,Jn)) where for each i, 1 <2 <n, the value of 7; is between 1 and &. For each 
extended execution a of M and each i, 1 <i <n, let 


{(,1),...,(¢,k)} if a; does not occur 
Ua)= 4 {(i,7)} if a; occurs and its first occurrence leads to U;,; 
fy otherwise. 


Then define GFIRST(S, EF) to be the function that associates with each probabilistic execution 
fragment H of M the set of extended executions a of Qy such that EA (Ui(argi!) x +--+ x 
Usavgl!)) #9. 

We illustrate the definition above by encoding the dice rolling example. In each tuple 
(a;, Uia,..., Ui) a; identifies the action of rolling the ith dice, k = 6, and for each j, U;; is 
the set of states where the i*® dice rolls 7. The set E identifies the set of outcomes that are 
considered to be good. In the case of the dices F is the set of tuples ((1,71),...,(”,Jn)) where 
for each number / between 1 and 6 there is at least one i such that j; = 1. The function U;(a) 
checks whether the 7*" dice is rolled and identifies the outcome. If the dice is not rolled, then, 
we allow any outcome as a possible one; if the dice is rolled and hits U;,;, then the outcome is 
(2,7); if the the dice is rolled and the outcome is not in any one of the sets U;,;’s, then there is 
no outcome (this case does not arise in our example). Then, an extended execution a of Qy 
is in the event GFIRST(S, E)(H) if at least one of the outcomes associated with avgé! is an 
element of E, i.e., if by choosing the outcome of the dices that are not rolled in apgé? all the 
six numbers appear as the outcome of some dice. 

Let p be the probability that by rolling n dices all the six numbers appear as the outcome 
of some dice. Then, the lemma below states that Py[GFIRST(S, E)(H)] > p for each H. 


Proposition 6.4.1 Let M be a probabilistic automaton. Let S be a set of n tuples {21,...,%n} 
where for each i, 1<i<n, a; is a tuple (aj, Uia,..., Ui) consisting of an action of M and k 
pairwise disjoint sets of states of M. Let the actions a; be all distinct. Let E be a set of tuples 
((1,91),---,(,Jn)) where for each i, 1 <i< n, the value of 3; is between 1 and k. For each 
i,j,l<i<n,1<j <k, let p;; be a real number between 0 and 1 such that for each transition 
(s,P) of M where Pla;] > 0, P[U;,;|a:] > pij, and let C be the collection of the p;js. Let Pe[E] 
be the probability of the event EF assuming that each experiment 1 is run independently, and 
that for each i a pair (t,7) is chosen with probability p;;. Then, for each probabilistic execution 


fragment H of M, Py[GFIRST(S, E)(H)| > Pe[F}. 


Proof. For each state q of H, each i € {1,...,n}, and each j € {1,...,k}, denote by Q(q, U;,;) 
the set {(a;,q’) € QF | Istate(q’) € U;;} of pairs where a; occurs and leads to a state of U;,;, 
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and denote by Q(q, U;) the set {(a;,q') € QF | Istate(q') ¢ U;U;,;} of pairs where a; occurs and 
none of the U;;s is reached. For each i € {1,...,n}, let O; be the set of states g of H such that 
no action a;, 1 <j <n, occurs in god, and PH ai] > 0. 

We prove the lemma by induction on n. If n = 1 then the result follows from Lemma 6.2.1 
(the event can be transformed into a new event with two outcomes); otherwise, 


Pu[GFIRST(S,E\ A= D> Dd) PulCy] ~~ Paid’) 


t€{1,....n} E90; (a5,q’/)EQ(9,0%) 


+ ( S- S- PE 02 Ps OPTS FGA) ) (6.23) 


FECL, k} (az,9JEQ(G, U5) 


where S; is obtained from S by removing the tuple (a;, Uja,...,U;,,), and £(;,;) is the set of tu- 


ples (1, 71),---,(@—1, gi-1), (@4+- 1, Jiqa),---, (5 Jn)) such that ((1, it). .(@—1, Ji-1), 9), G+ 
1, Jitu),---5(%Jn)) € BE. Let C; be obtained from C by removing all ‘the probabilities of the 
form pjj;, 1 <j <k. Then, by induction, 


Prog |GFIRST(Si, Fj) )(eq)] < (1 _ Fe, [Fuj))- (6.24) 
From the properties of conditional probabilities and the definition of C, 


Pe [EG] = PelE\(G, 3)I- (6.25) 


a 


Thus, by using (6.24) and (6.25) in (6.23), and by expressing P# [(a;, q’)] as P#[a;]P [(ai, q’)\ai], 
we obtain 


Py[GFIRST(S, EWA < S> D PalC Pai] PH (aig ail 
i€{1,....n} EO; (ai,qJEN(G,Ui) 
+ ( » PP [(ai, glad — PelEES)] »))- (6.26) 
JE{1,..4%} (ai,¢)E€Q(9,Ui,5) 


For each 2,7 and q, let pjj.q be PHO, U;,;)|a;]. Then, from (6.26), 


Py[GFIRST(S, EVA) < So SS Pa[C) PF [ail 


i€{1,...,.n} EO; 


[: Pilg | > rt REID). (6.27) 


gE{1,...,4} 


which becomes 


Py{GFIRST(S, E)(H)| 
< SO YDS Pal Pai] ( - > PEN hn) (6.28) 
1€(,-un} 191 jE nh} 
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after some simple algebraic simplifications. Using the same argument as in the proof of 
Lemma 6.2.1, for each 7,7 and each q, pj,j,q > pi. Thus, 


Py{GFIRST(S, E\(H)| 


< SPOS Pal PMlad{1- SO Pel apis | - (6.29) 


tE{1,....n} ¢EOs JE{1,...,4} 


Finally, observe that S7ie,1,..n} 2-qe0, Py[C,|P/ [ai] is the probability that some action a; 
occurs, and observe that jeq,.ny PelE|(¢, 7) pig = Pe[E]. Thus, 


Py{GFIRST(S, EA) <1 Pe[E] (6.30) 


6.4.2 A Generalized Coin Lemma 


All the coin lemmas that we have studied in this chapter share a common characteristic. Given 
a probabilistic execution fragment H, we identify n separate classes of random draws to observe. 
Each class can be observed at most once in every execution a of Qy, and if any class cannot 
be observed, then we allow for any arbitrary outcome. In this section we formalize this idea. 

Let H be a probabilistic execution fragment of a probabilistic automaton M. A coin-event 
specification for H is a collection C' of tuples (q, X,X1,...,X,%) consisting of a state of H, a 
subset X of Qn, and m pairwise disjoint subsets of X, such that the following properties are 
satisfied: 


1. for each state g of H there is at most one tuple of C' whose state is q; 


2. for each state g of H such that there exists a tuple of C with state q, there is no prefix q’ 
of q such that there exists a tuple (q’, X,X1,...,X,) in C and a pair (a,q") in X where 
q is a prefix of gq. 


The set C is the object that identifies one of the classes of random draws to be observed. For 
each transition tri? and each tuple (q, X, X1,..., Xx) of C, the set X identifies the part of tri? 
that is relevant for C, and the sets X1,..., X, identify some of the possible outcomes. The first 
requirement for C' guarantees that there is at most one way to observe what happens from a 
state g of H, and the second requirement states that along every execution of Q7 there is at 
most one place where C is observed. 

As an example, consider the observation of whether the first occurrence of an action a, 
which represents a coin flip, leads to head. Then C is the set of tuples (q, X, X1) where action 
a does not occur in grog! and PH al > 0, X is the set of pairs of QF where action a occurs, 
and X, is the set of pairs of X where the coin flips head. 

Let a be an extended execution of Oy, and let g be a state of H such that gq < a. We say 
that C occurs in a at q iff there exists a tuple (q, X, X1,..., X,) in C and a pair (a,q’) in X 
such that q’ < a. Moreover, if (a, gq’) € Xj, we say that C occurs in a at g and leads to X;. 

Two coin event specifications Cy and C> are said to be separate iff from every state q of 
H, if (q,X4, X14,..-, X14) is a tuple of Cy and (q¢, Xo, Xoa,...,X2,,) is a tuple of C2, then 
X19 X_ = 9. In other words, there is no interference between the observations of C; and the 
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observations of Cy. Let S = {C1,...,C,} be a set of pairwise separate coin-event specifications. 
For notational convenience, for each 7 € {1,...,n} and each state g of H such that there exists 
a tuple in C; with state g, denote such tuple by (q¢, X97, Xqi1,---;Xq,i,k) 

Let F be a set of tuples ((1,71),..-,(,jn)) where for each 7, 1 <i <n, the value of 3; is 
between | and &. For each extended execution a of Qy and each 21,1 <2< n, let 


{(4,1),...,(¢,k)} if C; does not occur in a 
Ua)= 4 {(i,7)} if C; occurs in a leading to Xqj,; 
otherwise. 


Then, define GCOIN(S,E)(H) to be the set of extended executions of Q7 such that EM 
(Ui(args") x +++ x Ux(argg')) # 0. 


Lemma 6.4.2 Let H be a probabilistic execution fragment of a probabilistic automaton M. Let 
S = {Cy,...,Cn} be a set of separate coin-event specifications for H. For each i,j, 1<i<n, 
1<j<k, let p;; be a real number between 0 and 1 such that for each i € {1,...,n} and each 
tuple (q, Xqji;Xq,ias+-+Xq,im) of Ci, PHX yi j|Xaq,il > pi; Let C be the collection of the p;;’s. 
Let Pe|E] be the probability of the event FE assuming that each experiment i is run independently, 
and for each i a pair (i, 7) is chosen with probability p;,;. Then, Py[GCOIN(S, E)(H)] > Pe[E]. 


Proof. For each state q of H and each i, 1 <2 <n, if there exists a tuple in C; with state q, 
then denote Xyi\ Ujesi,.k} Xq,i,7 by Xqi- For each i, 1 <i <n, let ©; be the set of states ¢ 
of H such that there exists a tuple with state g in C; and no coin-event Cj, 1 <7 <n, occurs 
: A 
in gogo - 

We prove the lemma by induction on n, using n = 0 for the base case. For n = 0 we assume 
that P[E] = 1 and that GCOIN(S, E)(H) = Qy. In this case the result is trivial. Otherwise, 


Py{GCOINS, EVM = SD YD PalGl| | SD Pitaa)) 
tE{1,...,.n} EO; (a,q’)EXqe 


+ ( S- S- PE 9) OCOTRT SE FTA) (6.31) 


FELL, KR} (QI) EXGi,5 


where Spq’ is obtained from S by removing C; and, for each 7 4 i, by transforming the set C; 
into {(qoq’, Xoq’, Xyoq',..., Xpoq’) | (q, X,X1,.-.,X-) € Cj,’ < gq}. Then, by induction, 


Prog |GCOIN (Seq’, Fj) )(eq)] < (1 _ Fe, [Fuj))- (6.32) 
From the properties of conditional probabilities and the definition of C, 
Pe [Ea] = PelLEG, I (6.33) 


Thus, by using (6.32) and (6.33) in (6.31), and expressing P?[(a,q')] as P#[X4;]P[(a, q')|Xq,i), 
we obtain 


Pu{GCOIN(S, EXD) < SDS Pel DPX | | YD PM Maa) Xyal 
tE{1,...,.n} EO; (a,q/)EX ga 


+ ( ~ Hosa REED) (6.34) 


JE{1,..4k} (4,q)EXgi5 
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For each i,j and q, let pj.j,q be PHXyi,j|Xq,il- Then, from (6.34), 


Pyl[GCOIN(S, EVM) < So SE Pal PX] 
tE{1,....n} qEO; 


[: — Piayg — 011 = Piskg) + ( Yo Ping - rele.) ; (6.35) 
je{1,..k} 


which becomes 
Py[GCOIN(S, E)(H)| 
< SSM Pal PPLX,5] [:- > PIE a) (6.36) 
tE{1,....n} ¢EOs JE{1,...,4} 


after some simple algebraic simplifications. From hypothesis, for each 7,7 and each q, pijjq > 
pi. Thus, 


Py[GCOIN(S, E)(H)| 
<D Y pwlg PF Xa ( _ > rete | (6.37) 
7E{1,...,.n} qeO; FELL, } 
Finally, observe that )7je41,....n} 2.qe€0; Py[C PP [Xai] is the probability that some C; occurs, 
and observe that ijeq1, ny PoLE|(4, J)|pi,g = Pe[E]. Thus, 
Py{GCOIN(S, E)(H)| < 1 — Po[E] (6.38) 


6.5 Example: Randomized Agreement with Stopping Faults 


In this section we analyze the Randomized Agreement algorithm of Ben-Or [BO83]. Its proof 
of correctness is an application of Lemma 6.4.2. The proof that we present in this section is not 
as detailed as the proof of the Dining Philosophers algorithm, but contains all the information 
necessary to fill in all the details, which we leave to the reader. 


6.5.1 The Problem 


Consider n asynchronous processes that communicate through a network of reliable channels 
(i.e., channels that deliver all the messages in the same order as they are received, and that 
never fail to deliver a message), and suppose that each process i starts with an initial value 
v; € {0,1}. Suppose that each process can broadcast a message to every other process in a 
single operation. Each process runs an algorithm that at some point may decide on one value 
of {0,1}. Each process decides at most once. The algorithm should be designed so that the 
following properties are satisfied. 


1. Agreement: all the processes that decide choose the same value. 
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2. Validity: if all the processes have the same initial value v, then v is the only possible 
decision value. 


3. f-failure termination: if at most f processes fail, then all the non-failing processes 
decide a value. 


We assume that a process fails by stopping, i.e., by failing to send messages to other processes 
from some point on. Since the processes are asynchronous, no processes can distinguish a slow 
process from a failing process. 

Unfortunately, it is known from [FLP85] that there is no deterministic algorithm for asyn- 
chronous processes that solves the agreement problem and guarantees 1-failure termination. 
Here we present the randomized algorithm of Ben-Or [BO83], which solves the agreement prob- 
lem with certainty, and guarantees f-failure termination with probability 1 whenever n > 3/f. 


6.5.2 The Algorithm 


Each process ? has local variables x, initially v;, and y, initially null, and executes a series of 
stages numbered 1,2,..., each stage consisting of two rounds. Each process runs forever, even 
after it decides. At stage st > 1, process zt does the following. 


1. Broadcast (first, st,v), where v is the current value of x, and then wait to obtain n — f 
messages of the form (first, st,*), where * stands for any value. If all the messages have 
the same value v, then set y := v, otherwise set y := null. 


2. Broadcast (second, st,v), where v is the current value of y, and then wait to obtain n— f 
messages of the form (second, st, *). There are three cases: 


(a) if all the messages have the same value v # null, then set x := v and perform a 
decide(v); operation if no decision was made already; 


(b) if at least n — 2 messages, but not all the messages, have the same value v £ null, 
then set  := v without deciding (the assumption n > 3f guarantees that there 
cannot be two different such values v); 


(c) otherwise, set x to 0 with probability 1/2 and to 1 with probability 1/2. 


The intuition behind the use of randomness is that at each stage, if a decision is not made yet, 
with probability at least 1/2” all the processes that choose a value at random choose the same 
good” value. Thus, with probability 1 there is eventually a stage where the processes that 
choose a value at random choose the same good value, and this leads to a decision. 

We now give an idea of the structure of the probabilistic automaton M that describes Ben- 
Or’s algorithm. Each process i has the two variables « and y mentioned in the description 
of the algorithm, plus a queue m; for each process 7 that records the unprocessed messages 
received from process j, initially null, a stage counter st, initially 1, a program counter pce, 
and a boolean variable decided that is set to true iff process 7 has decided already. There 
is a channel C;,; between every pair of processes. Each channel C;; is essentially a buffer 
like the buffer described in Chapter 3 (cf. Figure 3-1), whose inputs are actions of the form 
(first, st,v); and (second, st,v);, and whose outputs are actions of the form (first, st, v);,; and 
(second, st, v);,;. To broadcast a message (first, st, v), process 7 performs the action (first, st, v)j. 
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A message (first, st, v) is received by process ¢ from process 7 through the action (first, st, v);;. 
The definition of the transition relation of M is straightforward. 


6.5.3. The High Level Proof 


Agreement and validity are easy to prove and do not involve any probabilistic argument. 
Lemma 6.5.1 Ben-Or’s algorithm satisfies the agreement and validity conditions. 


Proof. We start with validity. Suppose that all the processes start with the same value v. 
Then it is easy to see that every process that completes stage 1 decides on v in that stage. This 
is because the only value sent or received by any process in the first round is v, and thus the 
only value sent or received by any process in the second round is v, leading to the decision of v. 

For agreement, suppose that some process decides, and let process 7 be the first process 
that decides. Let v and st be the value decided by process i and the stage at which process 
i decides, respectively. Then it must be the case that process i receives n — f (second, st, v) 
messages. This implies that any other process 7 that completes stage st receives at least n—2f 
(second, st, v) messages, since it hears from all but at most f of the processes that process 7 
hears from. This means that process 7 cannot decide on a value different from v at stage st; 
moreover, process j sets x := v at stage st. Since this is true for all the processes that complete 
stage st, then an argument similar to the argument for validity shows that any process that 
completes stage st + 1 and does not decide in stage st decides v at stage st + 1. | 


The argument for f-failure termination involves probability. We assume that all the processes 
but at most f are scheduled infinitely many times. Thus, let (fair be the set of adversaries for 
M such that for each probabilistic execution fragment H generated by an adversary of f-fair 
the set Qy contains only executions of M where at least n — f processes are scheduled infinitely 
many times. It is easy to check that f-fair is finite-history-insensitive. 

Let B be the set of reachable states of M; let F be the set of reachable states of MM where 
no process has decided yet and there exists a value st and a number 7 such that process 7 
received exactly n — f messages (first, st,*), and no other process has ever received more than 
n— f —1 messages (first, st, *); finally, let O be the set of reachable states of M where at least 
one process has decided. 

It is easy to show that 


B Their FUO. (6.39) 


Specifically, let a be an f-fair execution fragment of M starting from a reachable state s of MV, 
and let st be the maximum value of the stages reached by each process in s. Then, stage st + 1 
is reached eventually in a, and thus there is a state s’ in a where some process is the first one 
to receive n — f messages (first, st + 1,*). The state s’ is a state of FUO. 

In Section 6.5.4 we show that 


FO. (6.40) 
1/2” 


Thus, combining (6.39) and (6.40) with Theorem 5.5.2, and by using Proposition 5.5.6, we 
obtain 


BO. (6.41) 
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Finally, we need to show that in every f-fair execution where at least one process decides all 
the non-failing processes decide eventually. This is shown already in the second part of the 
proof of Lemma 6.5.1. 


6.5.4 The Low Level Proof 


In this section we prove the progress statement of (6.40) using the generalized coin lemma. 
Consider a state s of F, and let 7 be the process that has received n — f messages (first, st, v). 
Let A be an adversary of f-fair, and let H be prexec( M, A, s). 

For each 7, 1 < 7 < n, let C; be the set of triplets (¢, X,.X1) where q is astate of H such that 
process 7 is at stage st in Istate(q) and there is a non-zero probability that process 7 chooses 
randomly between 0 and 1 from qg, X is the set of pairs of QF where process j performs a 
transition, and X is defined as follows. Let s’ be Istate(q), and let v be a good value if at least 
f +1 of the messages (first, st, *) processed by process 7 have value v. We emphasize the word 
“processed” since, although each process can receive more that n—/f messages (first, st, *), only 
n— f of those messages are used (processed). 


1. If 0 is a good value, then let X, be the set of pairs of X where process 2 chooses 0; 


2. if 1 is a good value and 0 is not a good value, then let X1 be the set of pairs of X where 
process 7 chooses 1. 


Observe that in s’ there is at least one good value, and at most two values; thus, C; is well 
defined. It is easy to check that C,,...,C, are separate coin event specifications; more- 
over, for each 7, 1 < 7 < n, and each triplet (¢,X,X1) of Cj, PHIXG|X] = 1/2. Let 
E= {((1,1),(2,1),...,(n, l)}. From Lemma 6.4.2, Py[GCOIN((C1,...,Cn), £)()] > 1/2”. 

We are left with the proof that in each extended execution of GCOIN((C4,...,Cn), E)(H) 
all the non-faulty processes choose a value. More precisely, we show that the non-faulty pro- 
cesses complete stage st setting « to the same value v. Then, the second part of the proof of 
Lemma 6.5.1 can be used to show that all the non-faulty processes decide on v at the end of 
stage st + 1; in particular at least one process decides. We distinguish two cases. 


1. In s’ there is exactly one good value v. 


In this case every other process receives at least one copy of v during the first round of 
stage st, and thus y is set either to v or to null. Therefore, v is the only value that 
a process chooses by a non-random assignment at the end of stage st. On the other 
hand, if a process 7 chooses a value at random at the end of stage st, the definition of C; 
guarantees that the value chosen is v. Thus, every process that completes stage st sets 
Ei=v. 


2. In s’ there are two good values. 


In this case every process receives at least one copy of 0 and one copy of 1, and thus y 
is set to null. Therefore, each process chooses a value at random at the end of stage st. 
The definition of Cy,...,C, guarantees that every process that completes stage st sets 
e:= 0. 
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6.6 Example: The Toy Resource Allocation Protocol 


Lemma 6.4.2 can be used also to prove formally that the toy resource allocation protocol of 
Section 5.1 guarantees that, under any deterministic fair oblivious adversary (cf. Example 5.6.2 
for the definition of a fair oblivious adversary), process Mj, eventually gets a resource. This 
result can be extended to general oblivious adversaries by using the results about deterministic 
and randomized adversaries proved in Chapter 5 (cf. Proposition 5.7.11). 

Recall from Example 6.1.1 that we want to identify a coin event that expresses the following 
property: the first coin flip of M, after the first coin flip of M2 is different from the last coin 
flip of Mz before the first time M4, checks its resource after flipping. In the rest of the section 
we specify two coin event specifications Cy and C2. The specification Cy identifies the first coin 
flip of M, after the first coin flip of M2, while the specification C2 identifies the last coin flip of 
My) before the first time M4, checks its resource after flipping. 

Let H be a probabilistic execution fragment, generated by a deterministic fair oblivious 
adversary, such that the first state of g@/ is reachable in M. Let Cy, be the set of tuples 
(q, X, X1, X2) where 


1. gis astate of H such that Mb flips at least once in gogé!, M, does not flip in qogé! after 
the first time Mp flips, and M, flips from gq, 

2. X is the set QF, 

3. Xj, is the set of pairs of XY where M, flips head, 


4, Xo is the set of pairs of X where M, flips tail. 


Observe that Cy is a coin-event specification. Moreover, observe that for each tuple of C,, 
PHLX,|X] = 1/2 and PH[X3|X] = 1/2. Let C2 be the set of tuples (¢, X, Xi, X2) where 


1. gis a state of A such that either 


(a) M, does not flip in qoqé! after M2 flips, My flips from q, and there exists a state 
¢ > q such that My flips exactly once in q'>q and M, flips and checks its resource 
after flipping in qq, or 

(b) My, flips and does not check its resource after the first flip of M2 in qoge, Mz flips 
from q, and there exists a state gq’ > q such that My flips exactly once in q’>qg, My 
does not check its resource in q'eq, and M, checks its resource from q’, 


2. X is the set QF, 
3. Xj, is the set of pairs of XY where Mp flips head, 


4, X, is the set of pairs of X where Mp flips tail. 


Informally, Cz identifies the coin flip of Mz that precedes the point where Ad, checks the 
resource determined by Cy. Figure 6-4 illustrates graphically the two cases of the definition 
of C2. Observe that for each tuple of C2, P#[Xy|X] = 1/2 and P#![X2|X] = 1/2. Since H is 
generated by an oblivious deterministic adversary, then it is easy to verify that C> is a coin-event 
specification. The important point is to verify that Condition 2 of the definition of a coin event 
is satisfied; this is the point where the fact that an adversary is oblivious and deterministic is 
used. 
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Figure 6-4: The definition of Cg for the toy resource allocation protocol. 
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Figure 6-5: How C2 could not be a coin event specification. 


Example 6.6.1 (How C, could not be a coin event specification.) To give a rough idea 
of why Condition 2 does not fail, Figure 6-5 shows how Condition 2 could fail. Consider the 
execution of A, that is marked with +, and denote it by a; denote by a’ the other execution of 
Hf, that appears in the figure. The unfilled circles mark the points where a coin event speci- 
fication is observed. By following a from left to right we observe Cy and then we observe C9. 
The reason why we observe C2 the first time is that along a’ M, tests its resource. However, 
continuing to follow a, we observe C2 again because along a Mz tests its resource later. Using 
oblivious adversaries we are guaranteed that such a situation does not arise because if along a’ 
M;, tests its resource before Mo flips again, then the same property holds along a. 

The probabilistic execution Hz of Figure 6-5 illustrates how Condition 2 can fail by using 
randomized schedulers. After My, flips, the adversary chooses randomly whether to let My, test 
its resource (higher filled circle) or to let Mz continue. a 


Let EF be the set {((1,1)(2,2)),((1, 2),(2,1))}, which expresses the fact that Cy and C yield 
two different outcomes. It is easy to check that in every execution of GCOIN((Cj, C2), E)(H) 
M, eventually gets one resource. Thus, from Lemma 6.4.2, the probability that M, gets its 
resource in H is at least 1/4. Since H is a generic probabilistic execution fragment, then, under 
any deterministic fair oblivious adversary M, gets a resource eventually with probability at 
least 1/4. Since the set of deterministic fair oblivious adversaries is finite-history-insensitive, 
Lemma 5.5.6 applies, and we conclude that under any deterministic fair oblivious adversary My 
gets a resource eventually with probability 1. 


131 


6.7 The Partition Technique 


Even though the coin lemmas can be used to prove the correctness of several nontrivial algo- 
rithms, two of which have been illustrated in this chapter, there are algorithms for which the 
coin lemmas do not seem to be suitable. One example of such an algorithm is the random- 
ized algorithm for maximal independent sets of Awerbuch, Cowen and Smith [ACS94]; another 
example is the toy resource allocation protocol again. 


Example 6.7.1 (The coin lemmas do not work always) In Section 6.6 we have shown 
that the toy resource allocation protocol guarantees progress against fair oblivious adversaries; 
however, in Example 5.6.2 we have stated that the toy resource allocation protocol guarantees 
progress also against adversaries that do not know only the outcome of those coins that have 
not been used yet. Such a result cannot be proved using the coin lemmas of this chapter be- 
cause situations like those outlined in Example 6.6.1 arise. For example, after the first time M2 
flips, we could schedule My again and then schedule M, to test its resource only if Mo gets the 
resource Ry. 

Another way to obtain a situation where the coin lemmas of this chapter do not apply is to 
modify the second instruction of the resource allocation protocol as follows 


2. if the chosen resource is free, then get it, otherwise go back to 1. | 


Example 6.7.1 shows us that some other techniques need to be developed; it is very likely that 
several new techniques will be discovered by analyzing other algorithms. In this section we hint 
at a proof technique that departs considerably from the coin lemmas and that is sufficiently 
powerful to deal with the toy resource allocation protocol. We illustrate the technique with an 
example. 


Example 6.7.2 (The partition technique) Let A be a generic fair adversary for the toy 
resource allocation protocol that does not know the outcome of those coin flips that have not 
been used yet, and let H bea probabilistic execution generated by A. Assume for simplicity that 
A is deterministic; the result for a generic adversary follows from Proposition 5.7.11. Consider 
an element of Qy, and consider the first point gq where M, flips a coin (cf. Figure 6-6). The 
coin flipping transition leads to two states q, and q that are not distinguishable by A, which 
means that from qg, and q the adversary schedules the same process. If the process scheduled 
from gq, and gq is Mz, then the states reached from gy, are in one-to-one correspondence with the 
states reached from q;, since they differ only in the value of the coin flipped by M,. Figure 6-6 
illustrates the case where My flips a coin. Furthermore, two corresponding states are reached 
with the same probability. The one-to-one correspondence between the states reached form gp, 
and q is maintained until My, tests its chosen resource. 

Consider now a point where M, tests its resource. Figure 6-6 illustrates four of these points, 
denoted by dei, dhs G,2, and qn. If My fails to obtain the resource, it means that Mz holds 
that resource at that point. However, My holds the same resource in the corresponding state 
via the one-to-one correspondence M2, while M, tests the other resource. Thus, M1 succeeds 
in getting the chosen resource. (cf. states q¢1 and gq, of Figure 6-6. 

The bottom line is that we have partitioned the states where M, checks its resource in 
two sets, and we have shown that for each pair of corresponding states there is at least one 
state where M, succeeds in getting a resource. In some cases, like for states q@2, and qp.2 of 
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Figure 6-6: The partition technique. 


Figure 6-6, M, succeeds in getting its resource from both the corresponding states (M2 does 
not hold any resource). Thus, My gets a resource with probability at least 1/2. | 


6.8 Discussion 


To our knowledge, no techniques similar to our coin lemmas or to our partition technique were 
proposed before; however, similar arguments appear in several informal analysis of randomized 
algorithms. The idea of reducing the analysis of a randomized algorithm to the analysis of an 
ordinary pure nondeterministic system was at the base of the qualitative analysis techniques 
described in Sections 2.5.1 and 2.5.2. Here we have been able to apply the same idea for a 
quantitative analysis of an algorithm. 

In this chapter we have focused mainly on how to apply a coin lemma for the verification of 
a randomized algorithm; once a good coin event is identified, the analysis is reduced to verify 
properties of a system that does not contain randomization. We have carried out this last part 
using detailed operational arguments, which can be error prone themselves. However, since the 
problem is reduced to the analysis of a non-randomized system, several existing techniques can 
be used to eliminate our operational arguments. In [PS95] Segala and Pogosyants show how 
such an analysis can be carried out formally and possibly mechanized. 
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Chapter 7 


Hierarchical Verication Trace 
Distributions 


7.1 Introduction 


So far we have defined a model to describe randomized concurrent and distributed systems, 
and we have shown how to study the properties of a system by means of a direct analysis of its 
structure. A specification is a set of properties that an implementation should satisfy, and an 
implementation is a probabilistic automaton that satisfies the desired properties. 

Another approach to the analysis of a system considers an automaton as a specification itself. 
Then, an abstract notion of observation is defined on automata, and an automaton is said to 
be an implementation of another automaton iff there is a specific relation, usually a preorder 
relation, between their abstract observations. Examples of observations are traces [Hoa85, LV91] 
(cf. Section 3.2.3), and failures [Hoa85, BHR84]; in these two cases implementation is expressed 
by set inclusion. 


7.1.1 Observational Semantics 


Formally, an automaton A is associated with a set Obs(A) of observations, and a preorder 
relation R is defined over sets of observations (for example R can be set inclusion). Then, an 
automaton Aj, is said to implement another automaton Az, denoted by A; C Ag, iff Obs(A1) R 
Obs( Az). The function Obs() is called an observational semantics, or alternatively a behavioral 
semantics; in the second case the observations are thought as the possible behaviors of an 
automaton. 

The methodology based on preorder relations is an instance of the hierarchical verification 
method: a specification, which is usually very abstract, can be refined successively into less 
abstract specifications, each one implementing the more abstract specification, till the actual 
implementation is obtained. Figure 7-1 gives an example of a specification that is refined two 
times to build the actual implementation. Of course it is implicitly assumed that the relevant 
properties of a system are only those that are preserved by the chosen implementation relation. 
Thus, given a relation, it is important to understand what properties it preserves. Coarse 
relations may not preserve all the relevant properties, but they are usually easy to verify, i.e., it 
is usually easy to establish whether such a relation holds; finer relations that preserve exactly the 
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Figure 7-1: Refinement of a specification. 
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Figure 7-2: Modular design. 


relevant properties are usually difficult to characterize and verify; other relations that preserve 
all the relevant properties and that are easy to verify are usually too fine, i.e., they distinguish 
too much. Some tradeoff is necessary. 


7.1.2 Substitutivity and Compositionality 


When the size of a problem becomes large, it is common to decompose the problem into simpler 
subproblems that are solved separately. Figure 7-2 gives an example. A large specification S$ is 
decomposed into several subcomponents M,,...,M, that interact together to implement S$. For 
example, a complex computer system can be described by the interaction of a central processor 
unit, a memory unit, and an Input/Output unit. Then, each subcomponent specification M; is 
given to a development team that builds an implementation M/. Finally, the implementations 
are put together to build an actual implementation of $. This kind of approach is called modular 
design; however, in order to guarantee the soundness of modular design, we need to guarantee 
that an implementation works properly in every context where its specification works properly, 
i.e., our implementation relation must be preserved by parallel composition (i.e., it must be a 
precongruence ). This property is called substitutivity of a preorder relation, and constitutes one 
of the most important properties that an implementation relation should satisfy. 

A property that is strictly related to the substitutivity of CL is called compositionality 
of Obs(). That is, there is an operator || defined on pairs of sets of observations such that 
Obs(Aj||Az) = Obs(A,)||Obs( Ag). Compositionality and substitutivity are used interchange- 
ably when talking informally about concurrent systems, and it is easy to get confused by the 
meanings of the two terms. To clarify every doubt, here is how the two concepts are related. 
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Theorem 7.1.1 Let Obs() be an observational semantics, R be an equivalence relation over 
sets of observations, and let, for each set x of observations, [|p be the equivalence class of 
z under R. Let Ay = Ag iff Obs(A1) R Obs( Ag). Then the following two statements are 


equivalent. 
1. = is substitutive, i.e., if Ay = Ag then for each A3, Ay||A3 = Aal| As; 


2. Obs() is compositional, i.e., there exists an operator || on equivalence classes of observa- 


tions such that [Obs(Ay||Az)|r = [Obs( Aj )]R||[Obs( Ar) ]R- = 


If # is set equality, then we can remove the equivalence classes from the second statement 
since each set of observations is an equivalence class. The substitutivity of a preorder relation 
is stronger than the substitutivity of its kernel equivalence relation, since the direction of the 
inequality must be preserved under parallel composition. For this reason our primary concern 
in this chapter is the substitutivity of the implementation relation. 


7.1.3 The Objective of this Chapter 


In this chapter we study the simplest implementation relation based on observations, i.e., trace 
inclusion, and we extend the corresponding precongruence to the probabilistic framework. The 
trace preorder constitutes the basis for several other implementation relations and is known to 
preserve the safety properties of a system [AS85]. Roughly speaking, a safety property says that 
“something good holds forever” or that “something bad does not happen”. The trace preorder 
is important for ordinary automata for its simplicity and for the availability of the simulation 
method |[LT87, Jon91, LV91] (cf. Chapter 8), which provides several sufficient conditions for 
the trace preorder relation to hold. Other relations, based either on failures [Hoa85, BHR84] 
or on any other form of enriched traces, can be obtained by following the same methodology 
that we present here. 

In the probabilistic framework a trace is replaced by a trace distribution, where the trace 
distribution of a probabilistic execution fragment H is the distribution over traces induced by 
Puy, the probability space associated with H. The trace distribution preorder is defined as 
inclusion of trace distributions. 

Unfortunately, the trace distribution preorder is not a precongruence (cf. Example 7.4.1), 
which in turn means that the observational semantics based on trace distributions is not com- 
positional. A standard approach in this case is to define the trace distribution precongruence 
as the coarsest precongruence that is contained in the trace distribution preorder; then, in 
order to have a compositional observational semantics that captures the trace distribution pre- 
congruence, an alternative, more operational and constructive characterization of the trace 
distribution precongruence is derived. We give an alternative characterization of the trace dis- 
tribution precongruence by exhibiting a context, called the principal context, that distinguishes 
two probabilistic automata whenever there exists a distinguishing context. This leads to the 
notion of a principal trace distribution, which is a trace distribution of a probabilistic automaton 
in parallel with the principal context; the trace distribution precongruence can be characterized 
alternatively as inclusion of principal trace distributions. 

Several other characterizations of the trace distribution precongruence could be found, pos- 
sibly leading to different observational semantics equivalent to the principal trace distribution 
semantics. Further experience with each one of the alternative semantics will determine which 
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Figure 7-3: Trace distribution equivalent probabilistic automata. 


one is more useful. One of the problems with the principal trace distribution characterization 
is that, although from Theorem 7.1.1 there exists an operator || defined on principal traces, 
the definition of || is not simple. For ordinary automata the traces of a parallel composition 
of two automata are exactly those sequences of actions that restricted to each component give 
a trace of the component. This property does not hold for principal trace distributions (cf. 
Example 7.4.1). It is desirable to find a semantics that characterizes the trace distribution 
precongruence and for which the corresponding parallel composition operator has a simple 
definition; however, it is not clear whether such a semantics exists. 


7.2. Trace Distributions 


Let H be a probabilistic execution fragment of a probabilistic automaton M, and let f be a 
function from Qy to Q = eat(H)*Ueart(H)” that assigns to each execution of Qy its trace. The 
trace distribution of H, denoted by tdistr(H), is the probability space completion((Q, F, P)) 
where F is the o-field generated by the cones C'g, where (3 is a finite trace of H, and P = f(Py). 
Observe that, from Proposition 3.1.4, f is a measurable function from (QH, Fz) to (Q, F), since 
the inverse image of a cone is a union of cones. Denote a generic trace distribution by D. A trace 
distribution of a probabilistic automaton is the trace distribution of one of the probabilistic 
executions of M. Denote by tdistrs(M) the set of the trace distributions of a probabilistic 
automaton M. 

It is easy to see that trace distributions extend the traces of ordinary automata: the trace 
distribution of a linear probabilistic execution fragment a is a distribution that assigns proba- 
bility 1 to trace(a). 

Given two probabilistic execution fragments H, and Ho, it is possible to check whether 
idistr(H,) = tdistr(H2) just by verifying that Praisi(H,)[Ca] = Praistr(H)lCa] for each finite 
sequence of actions 3. This is an easy consequence of the extension theorem (cf. Theorem 3.1.2). 


Example 7.2.1 (Reason for the definition of 2) The reader may wonder why we have 
not defined 2 to be trace(Qy). This is to avoid to distinguish two trace distribution just be- 
cause they have different sample spaces. Figure 7-3 illustrates the idea. The two probabilistic 
automata of Figure 7-3 have the same trace distributions; however, the left probabilistic au- 
tomaton has a probabilistic execution where the trace a occurs with probability 0, while the 
right probabilistic automaton does not. Thus, by defining the sample space of tdistr( H) to be 
trace(Qy), the two probabilistic automata of Figure 7-3 would be distinct. In Chapter 8 we 
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define several simulation relations for probabilistic automata, and we show that they are sound 
for the trace distribution precongruence; such a result would not be true with the alternative 
definition of a trace distribution. | 


Prefixes 


The notion of a prefix for traces can be extended to the probabilistic framework by following 
the same idea as for the notion of a prefix defined on probabilistic executions (cf. Section 4.2.6). 
A trace distribution D is a prefix of a trace distribution D’, denoted by D < D’, iff for each 
finite trace 3, Pp[Cg] < Pp: [Cg]. Thus, two trace distributions are equal iff each one is a prefix 
of the other. 


Lemma 7.2.1 Let Hy and Hy» be two probabilistic execution fragments of a probabilistic au- 
tomaton M. If Hy < Ho, then tdistr(H,) < tdistr( H2). | 


Action Restriction 


Similarly to the ordinary case, it is possible to define an action restriction operator on trace 
distributions. Let D = (Q,F,P) be a trace distribution, and let V be a set of actions. Then 
the restriction of D to V, denoted by D [ V, is the probability space completion((Q', F’, P’)) 
where 0'= 0} V, F’ is the o-field generated by the sets of cones of 2’, and P’ is the inverse 
image of P under the function that restricts traces to V. 


Lemma 7.2.2 Let D be a trace distribution. Then (D } Vi) | V2 =D (Vin V2). 


Proof. This is a direct consequence of the fact that restricting a trace to V; and then to V2 is 
equivalent to restricting the same trace to V; V2. Formally, - f (Vin V2) = (- f Va)o(-— V1). a 


Finally, we want to show that, if M = Mj,||Mo, then the projection of a trace distribution of 
M onto M, and Mo is a trace distribution of M1 and Mo, respectively. Formally, 


Proposition 7.2.3 If D € tdistrs(M,||M2), then D [ acts(M;) € tdistrs(M;), 7 = 1,2. 


The converse of Proposition 7.2.3 is not true; an illustrating example is given in Section 7.4 
(cf. Example 7.4.1). The rest of this section is dedicated to the proof of Proposition 7.2.3. We 
start with a definition of an internal trace distribution, which is a trace distribution that does 
not abstract from internal actions. 

Let a be an execution of a probabilistic automaton M. The internal trace of a, denoted 
by itrace(a), is the subsequence of a consisting of the actions of M. Let H be a probabilistic 
execution fragment of M, and let f be a function from Qy to Q = acts(H)* U acts(H)” that 
assigns to each execution of Oy its internal trace. The internal trace distribution of H, denoted 
by itdistr(H), is the probability space completion((Q,F,P)) where F is the o-field generated 
by the cones of 2, and P = f(Py). Observe that, from Proposition 3.1.4, f is a measurable 
function from (Qy, Fy) to (Q,F). Denote a generic internal trace distribution by D. Denote 
the set of internal trace distributions of a probabilistic automaton M by itdistrs(M). 


Lemma 7.2.4 Let H be a probabilistic execution fragment of a probabilistic automaton M. 


Then, tdistr( H) = itdistr(H) } ext(H). 
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Proof. This is a direct consequence of the fact that the set of executions of H whose trace 
contains a given ( is the set of executions of H whose internal trace restricted to the external 
actions of H contains 3. Formally, trace(-) = itrace(-) 0 (- f et(H)). = 


Lemma 7.2.5 Let H be a probabilistic execution fragment of M,||Mz, where My and My are 
two compatible probabilistic automata. Then itdistr( H|M;) = itdistr( H) | acts(M;), t= 1,2. 


Proof. Let P denote itdistr(H|M;), and let P’ denote itdistr(H) | acts(M;). We need to 
show that for each finite internal trace 6, P[C'g] = P’[Cg]. Let P” denote itdistr(H). From the 
definition of an internal trace, 


P(Ce\ = Parle € Qua, | 8 < ttrace(a)). (7.1) 
From the definition of P’ and P”, 
P'C,] = PB € Q" | 8 < B'  acts(.M;)). (7.2) 


From the definition of itdistr(H) and (7.2), 

PCs] = Pula € Qy | 8 < itrace(a) | acts(M;)). (7.3) 
Thus, from (7.1) and (7.3), we need to show that 

Prru,la € Quy; | 8 < trace(a)] = Pyla € Qy |B < itrace(a) [ acts(M;)}. (7.4) 


By using a characterization of the involved events as a disjoint union of cones, and by rewriting 
Equation 7.4 accordingly, we obtain 


Puyuil U C4] (7.5) 
q€states(H [M,)|ttrace(q)=8,lact(q)=lact(Z) 
= Py UJ Cy). 
q€states(H)|itrace(q) acts(M;)=6,lact(q)=lact(G) 
Observe that for each ¢ € states(H) such that itrace(q) [ acts(M;) = @ and lact(q) = 
lact(3), the state g[M; is a state of H[M; such that itrace(q[M;) = 6 and lact(q[M;) = 
lact(3). Moreover, the states q of the left expression of (7.5) are partitioned by the relation 


that relates g and q’ whenever q[M; = q'[M;. Thus, if we show that for each trace 3 and each 
q € states(H|M;) such that itrace(q) = 6 and lact(q) = lact(9), 


Puro, [C4] = PH|Ugreg| H|lact(q")=lact(p)Ca'|: (7.6) 
Equation (7.5) is proved. Observe that 
Pr|Ugrestates(H)|q![Mixajlact(q!)=lact(3)C 7’ = S- Pr[Cq), (7.7) 


( 
since {q’ € states(H) | q'|M; = ¢q, lact(q’) = lact(3)} = min(q|H). Thus, Equation (7.6) 
becomes 


PuulCg= >> Pauly, (7.8) 
q'Emin(q|H) 


which is true from Proposition 4.3.5. | 
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Lemma 7.2.6 Let H be a probabilistic execution fragment of M,\|M2z, where My and My are 
two compatible probabilistic automata. Then tdistr( H|M;) = tdistr(H) [ acts(.M;). 


Proof. From Lemma 7.2.4, 

tdistr( H|M;) = itdistr( H[M;) [ ext(M;). (7.9) 
From Lemma 7.2.5 and (7.9), 

tdistr( H[M;) = (itdistr( 1) [ acts(.M;)) | ext(.M;). (7.10) 


From Lemma 7.2.2 and (7.10), 


( 
) 
( 
) 


tdistr( H[M;) = (itdistr( 1) [ ext(H1)) | acts(.M;). (7.11) 
From Lemma 7.2.4 and (7.11), 
tdistr( H|M;) = tdistr(H) [ acts(.M;), (7.12) 


which is what we needed to prove. | 


Proof of Proposition 7.2.3. Let D € tdistrs(M,||Mz). Then there exists a probabilis- 
tic execution H of M;||M2 such that tdistr(H) = D. From Proposition 4.3.4, H/M; is a 
probabilistic execution of M;. From Lemma 7.2.6, tdistr(H|[M;) = D | acts(M;). Thus, 
D } acts(M;) € tdistrs(.M;). = 


7.3. Trace Distribution Preorder 


Once trace distributions are defined, the trace distribution preorder can be defined as trace 
distribution inclusion. Formally, let M1, 44. be two probabilistic automata with the same 
external action signature. The trace distribution preorder is defined as follows. 


M, Cp Mz iff tdistrs(. My) C tdistrs( M2). (7.13) 


The trace distribution preorder is a conservative extension of the trace preorder of ordinary 
automata, and it preserves properties that resemble the safety properties of ordinary automata 
[AS85]. Here we give some examples of such properties. 


Example 7.3.1 The following property is preserved by the trace distribution preorder. 


“After some finite trace G has occurred, then the probability that some other trace 
3 occurs, is not greater than p.” 


In fact, suppose that M, Ep Mo, and suppose that Mo satisfies the property above, while 
M, does not. Then there is a trace distribution of M, where the probability of 6’ after 6 
conditional to 9 is greater than p. Since My, Ep Mg, there is a trace distribution of Mz where 
the probability of 3’ after G conditional to @ is greater than p. This contradicts the hypothesis 
that Moz satisfies the property above. Observe that the property above would still be preserved 
if we replace @’ with a set of traces. | 
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Example 7.3.2 The following property is preserved by the trace distribution preorder. 


“In every computation where infinite external activity occurs with probability 1, if 
a finite trace 3 occurs, then the probability that some other trace 3’ occurs after 3 
given that 9 occurs is at least p.” 


A more concrete instantiation of the property above is “under the hypothesis that a distributed 
system never deadlocks, every request of service eventually gets a response with probability at 
least p”. This property is definitely more interesting than the property of Example 7.3.1 since it 
involves a progress statement, one of the property of key interest for the analysis of randomized 
distributed algorithms. Thus, if in a system it is always possible to avoid a deadlock, under 
the assumption that we always schedule a transition and under the condition that no infinite 
internal computation is possible, the property above guarantees progress. However, in order to 
be sure that if M@, Ep M2 and Mp satisfies the property above then M, guarantee progress, we 
need to make sure that from every state of Mo it is possible to avoid deadlock and there is no 
possibility of infinite internal computation. Such a property must be verified separately since it 
is not guaranteed by the trace distribution preorder. Fortunately, there are several cases (e.g., 
n processes running in parallel that communicate via shared memory) where it is easy to verify 
that it is always possible to avoid a deadlock. 

To prove that the property above is preserved, suppose that M1, Lp Mo, and suppose that 
Mz satisfies the the property above, while Md, does not. Then there is a trace distribution of MM, 
with infinite external computation where the probability of @’ after G conditional to @ is greater 
than p. Since My Cp Mg, there is a trace distribution of M2 with infinite external computation 
where the probability of 9’ after G conditional to @ is greater than p. This contradicts the 
hypothesis that M2 satisfies the property above. | 


Example 7.3.3 The following property is preserved by the trace distribution preorder. 


“In every computation where infinite external activity occurs with probability 1, if a 
finite trace 3 occurs, then, no matter what state is reached, a trace 3! occurs ofter 
GB with probability at least p.” 


A more concrete instantiation of the property above is “under the hypothesis that a distributed 
system never deadlocks, if a process has requested a service (3), then, no matter what state is 
reached, either the service has received a positive acknowledgment already (/3’), or a positive 
acknowledgment will be received eventually with probability at least p”. This property is pre- 
served by the trace distribution preorder since it is equivalent to the property of Example 7.3.2 
with p = 1 (cf. Proposition 5.5.5 to have an idea of why this is true). | 


Essentially, the rule of thumb to determine what properties can be guaranteed to be preserved 
under the trace distribution preorder is the following: express the property of interest as a 
property ¢ of the trace distributions of a probabilistic automaton M plus a condition w on the 
structure of M. If My Ep Mo, then the trace distributions of A, satisfy the property ¢. Thus, 
if we know that Mo satisfies the property of interest, it is enough to verify separately that M, 
satisfies 4 in order to be guaranteed that also My, satisfies the property of interest. 
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Figure 7-4: The trace distribution preorder is not a precongruence. 
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Figure 7-5: A probabilistic execution of Mg||C. 


7.4 Trace Distribution Precongruence 


Although the trace distribution preorder preserves some properties that are useful for the anal- 
ysis of randomized distributed systems, the trace distribution preorder is not a precongruence, 
and thus it does not allow us to use modular analysis. 


Example 7.4.1 (The trace distribution preorder is not substitutive) Consider the two 
probabilistic automata M, and My of Figure 7-4. It is easy to check that My and Mp) have 
the same trace distributions. Consider now the context C’ of Figure 7-4. Figure 7-5 shows a 
probabilistic execution of M2||C where there is a total correlation between the occurrence of 
actions d and f and actions e and g. Such a correlation cannot be obtained from M4||C, since 
the choice between f and g must be resolved before knowing what action among d and e is 
chosen probabilistically. Thus, M,||C and M2||C do not have the same trace distributions. ™ 


This leads us to the definition of the trace distribution precongruence, denoted by Epc, as the 
coarsest precongruence that is contained in the trace distribution preorder. This definition of the 
trace distribution precongruence is not constructive, and thus it is difficult to understand what 
we have defined. Furthermore, we do not have any observational semantics that characterizes 
the trace distribution precongruence. In Section 7.5 we give an alternative characterization 
of the trace distribution precongruence that gives a better idea of the relation that we have 
defined. Here we give some examples of properties that are preserved by the trace distribution 
precongruence and that are not preserved by the trace distribution preorder. 


Example 7.4.2 The following property is preserved by the trace distribution precongruence 
but not by the trace distribution preorder. 
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“After some finite trace B has occurred, no matter what state is reached, the prob- 


ability that some other trace 3' occurs from the state reached is not greater than 
” 


p. 
This property is not preserved by the trace distribution preorder since trace distributions cannot 
detect all the points where we may start to study the probability of 3’ to occur. However, this 
task is possible with the help of an external context. We use a context C that performs a fresh 
action o and then stops. 

Suppose that M, Epc M2 and suppose that M2 satisfies the property above, while M, 
does not. Then there is a probabilistic execution H, of M, where some state g is reached after 
the occurrence of @, and the probability that 6’ occurs from q is greater than p. Consider a 
probabilistic execution Hj of M,||C such that Hj[M, = H, and such that action o is scheduled 
exactly from the minimal state q’ such that q’[M, = q. Then, o occurs always after 3, and 
the conditional probability of 3’ after o given that o occurred is greater than p in the trace 
distribution of Hj. Since M; Epc Mb, then there is a probabilistic execution H} of M2||C 
whose trace distribution is the same as the trace distribution of H5. This means that there is at 


least one state q” in H4, reached immediately after the occurrence of 0, where the probability 
that 2’ occurs from q” in Hj is greater than p. Consider H}[M2, and change its transition 
relation to obtain a probabilistic execution H 2 such that H2>(q"[M2) = (H5[M2)e(q"[ Mo). 
Then the probability that 8’ occurs from q"”[Mz in Hy is greater than p. Moreover, ( has 
occurred when g[ Mg is reached. This contradicts the hypothesis that M2 satisfies the property 
above. | 


Example 7.4.3 The following property is preserved by the trace distribution precongruence 
but not by the trace distribution preorder. 


“In every computation where infinite external activity occurs with probability 1, if a 
finite trace B occurs, then, no matter what state is reached, if another trace 3" has 
not occurred yet after 3, then a trace 3’ occurs with probability at least p.” 


A more concrete instantiation of the property above is “under the hypothesis that a distributed 
system never deadlocks, if a process has requested a service (3) and has not received yet a 
refusal (3) then, no matter what state is reached, a positive acknowledgment (’) will be 
received eventually with probability at least p”. Observe that the main difference from the 
property of Example 7.3.3 is in the use of 3”. The presence of 3” does not guarantee that 3’ 
occurs with probability 1. 

Even in this case in the proof we use a context C with a fresh action 0. Suppose that 
M, Epc Mz and suppose that Mp2 satisfies the property above, while 44, does not. Then there 
is a probabilistic execution H, of M, where infinite external activity occurs such that there is a 
state gq of H, that is reached after the occurrence of 3 and before the occurrence of 3”, and such 
that the probability that 3’ occurs from q is smaller than p. Consider a probabilistic execution 
Hy of My||C such that Hj[M, = Hy and such that action o is scheduled exactly from the 
minimal state gq’ such that q’/[ M1 = q. Then, o occurs always after 3 and before 3” occurs after 
G, and the conditional probability of 9’ after o given that o occurred is greater than p in the 
trace distribution of Hj. Since M; Epc Mg, then there is a probabilistic execution Hf of Ma||C 
whose trace distribution is the same as the trace distribution of H5. This means that there is at 
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Figure 7-6: The principal context (left) and the simple principal context (right). 


least one state q” in H5, reached immediately after the occurrence of 0, where the probability 
that 2’ occurs from q” in Hf is smaller than p. Consider H4/M2, and change its transition 
relation to obtain a probabilistic execution Hz such that H2>(q"[M2) = (HS[M2)r(q"[ M2). 
Then the probability that 6’ occurs from q"”{Mz in Hy is smaller than p. Moreover, has 
occurred when g[ Mp is reached and similarly 6” has not occurred after the occurrence of 3. 
This contradicts the hypothesis that M2 satisfies the property above. | 


7.5 Alternative Characterizations of the Trace Distribution 
Precongruence 


In this section we give an alternative characterization of the trace distribution precongruence 
that is easier to manipulate. We define a principal context, denoted by Cp, and we show that 
there exists a context C that can distinguish two probabilistic automata M, and Mo iff the 
principal context distinguishes My and Mo. 


7.5.1 The Principal Context 


The principal context is a probabilistic automaton with a unique state and three self-loop tran- 
sitions labeled with actions that do not appear in any other probabilistic automaton. Two 
self-loop transitions are deterministic (Dirac) and are labeled with action left and right, respec- 
tively; the third self-loop transition is probabilistic, where one edge leads to the occurrence of 
action pleft with probability 1/2 and the other edge leads to the occurrence of action pright 
with probability 1/2. Figure 7-6 shows the principal context. 

The principal context is not a simple probabilistic automaton; however, since it does not 
have any action in common with any other probabilistic automaton, the parallel composition 
operator can be extended trivially: no synchronization is allowed. Alternatively, if we do not 
want a non-simple context, we can replace the principal context with the simple principal 
contert, represented in Figure 7-6, as well. In this case we need to assume that also action start 
does not appear in any other probabilistic automaton. The main theorem is the following. 


Theorem 7.5.1 M, Cpe Me iff My ||Cp Cp M3\||Cp. | 


As a corollary we obtain an alternative characterization of the trace distribution precongruence 
and a compositional observational semantics for probabilistic automata. A principal trace distri- 
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bution of a probabilistic automaton M is a trace distribution of M||Cp. Denote by ptdistrs(M) 
the set tdistrs(.M||Cp). 


Corollary 7.5.2 My Epc M2 iff ptdistrs(M,) C ptdistrs( M2). | 


The fact that the principal context is not a simple probabilistic automaton may appear to 
be confusing. Here we shed some light on the problem. First of all, in Chapter 4 we have 
defined parallel composition only for simple probabilistic automata; in this section, in order to 
account for the principal context, we have extended parallel composition to pairs of probabilistic 
automata, not necessarily simple, that do not have any action in common. This raises an 
immediate question: is the trace distribution precongruence defined based solely on contexts 
that are simple probabilistic automata or is it defined based on any compatible context according 
to the new extended parallel composition? The answer to this question, as it will become clear 
from the proof of Theorem 7.5.1, is that it does not matter because the two definitions are 
equivalent. That is, if there is a non-simple context that distinguishes two simple probabilistic 
automata M; and Mo, then the simple principal context distinguishes Md, and My as well. 

Our choice of the principal context is just stylistic since it contains less structure than 
the simple principal context. The reader should keep in mind that there are infinitely many 
contexts with the same properties as the principal and the simple principal contexts; any one 
of those contexts can be chosen to give an alternative characterization to the trace distribution 
precongruence. 


7.5.2 High Level Proof 


The rest of this section is dedicated to the proof of Theorem 7.5.1. The proof is structured 
in several steps where at each step a generic distinguishing context C’ is transformed into 
a simpler distinguishing context C’. The proof of each transformation step is structured as 
follows. Given a distinguishing context C’ for My Ep Mo, build a simpler context C’. Suppose 
by contradiction that C’ is not a distinguishing context and consider a trace distribution D of 
M,\|C that is not a trace distribution of M||C. Let H, be a probabilistic execution of M,||C 
such that tdistr(H,) = D. Transform Hj into a probabilistic execution Hj of Mj||C’, and show 
that if there is a probabilistic execution H} of Mo||C’ such that tdistr(H$) = tdistr(H{), then 
HS, can be transformed into a probabilistic execution Hz of M2||C such that tdistr( Hz) = D. 
This leads to a contradiction. 
The high level proof of Theorem 7.5.1 is then the following. 


=>: Assuming that the principal context distinguishes M, and M2, we show that the simple 
principal context distinguishes M, and Mo. 


<=: We consider a generic context C’ that distinguishes M, and M2, and we transform it into 
the principal context, showing that the principal context distinguishes M, and Mz. The 
transformation steps are the following. 
1. Ensure that C does not have any action in common with M, and M2 (Lemma 7.5.3); 
2. Ensure that C does not have any cycles in its transition relation (Lemma 7.5.4); 


3. Ensure that the branching structure of C' is at most countable (Lemma 7.5.5); 
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4. Ensure that the branching structure of C' is at most binary (Lemma 7.5.6); 


5. Ensure that the probabilistic transitions of C lead to binary and uniform distributions 
(Lemma 7.5.7); 


6. Ensure that each action of C is external and appears exactly in one edge of the 
transition relation of C' (Lemma 7.5.8); 


7. Ensure that each state of C enables two deterministic transitions and one probabilis- 
tic transition with a uniform binary distribution (Lemma 7.5.9); 


8. Rename all the actions of the context of 7 according to the action names of the 
principal context and then collapse all the states of the new context into a unique 
state, leading to the principal context (Lemma 7.5.10). 


7.5.3. Detailed Proof 


Lemma 7.5.3 Let C be a distinguishing context for two probabilistic automata M, and My). 
Then there exists a distinguishing context C’ for My, and Mz with no actions in common with 
M, and My. C’ is called a separated context. 


Proof. The context C’ is built from C’ be replacing each action a in common with M, and Mg, 
called a shared action, with two new actions a1, a2, and by replacing each transition (c,a,P) of 
C with two transitions (c, a1, c’) and (c’,a2,P), where c’ denotes a new state that is used only 
for the transition (c,a,P). We denote c’ also by C(c,a,p) When convenient. We also denote the 
set of actions of the kind a, and az by Vy and Va, respectively. 

Let D be a trace distribution of M,||C that is not a trace distribution of M2||C. Consider a 
probabilistic execution Hy of M,||C such that tdistr(H,) =D, and consider the scheduler that 
leads to Hy. Apply to M,||C’ the same scheduler with the following modification: whenever a 
transition ((s1,¢),a,P ® P) is scheduled in M,||C, schedule ((s1,¢), a1, P((s1,¢’))), where c’ is 
C(c,a,P)> followed by ((s1,¢’), a, Pi@D(c’)), and, for each s{ € 1, followed by ((s{, ¢”), a2, Ps) ® 
P). Denote the resulting probabilistic execution by Hj and the resulting trace distribution by 
D’. Then, 


D' | acts(M,||C) = D. (7.14) 


To prove (7.14) we define a new construction, called collapse and abbreviated with clp, to be 
applied to probabilistic executions of M;||C’, 7 = 1,2, where each occurrence of a shared action 
ais followed immediately by an occurrence of its corresponding action a. 

Let H’ be a probabilistic execution of M;||C’ where each occurrence of a shared action a is 
followed immediately by an occurrence of its corresponding action a2. For convenience denote 
clp(H') by H. A state q of H’ is closed if each occurrence of a shared action a is followed 
eventually by an occurrence of the corresponding action a2. For each closed state g of H’, let 
clp(q) be obtained from q as follows: each sequence 


(So, €o)a1(So, Ctr )Ta( 82, Ctr) aan Th( Sk; Ctp als, Ctr )a2(8, c) 


is replaced with 


(So, €o)T2(S2, Co) . -Th( Sk, co)a(s, c), 
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and each sequence 
(S0, €o)a1(S1, Ctr) T2( $2, Ctr) ++ Th(Sk, Ctr) 
occurring at the end of ¢ is replaced with 
(80, Co)T2(S2, Co) +++ Tk( Sk, Co). 
Define 
states(H) = {celp(q) | q € states( H’), closed(q)}. (7.15) 


Let (¢, P) be a restricted transition of H’ where q is a closed state, and suppose that no action 
of Vi U Vg occurs. Consider a pair (a,q’) of 2. If a is not a shared action, then let 


Prag) = D((a, elp(q’))): (7.16) 
if a is ashared action, then let 

Yaar = {(a,clp(q’)) | (a2,q”) € QE", (7.17) 
and for each (a, q!") € Q(a,g), let 

Pag lag”)| = Pyar x elp'(d")), (7.18) 


where for each state q of H, clp~'(q) is the set of closed states q! of H’ such that clp(q') = q. 
The transition clp((q,P)) is defined to be 


clp((q,P)) = (sin. » Pod Pn) (7.19) 


(a,q/)EQ 


For the transition relation of H, consider a state q of H Let min(clp—'(q)) be the set of minimal 
states of clp~'(q) under prefix ordering. For each state ¢ € clp~'(q), let 


“clp-1(q) A Py (Cal 
D; SE (7.20) 
" yo glemin(elp-1(q)) PHC a] 
The transition enabled in A from gq is 
So pt? PH Tacts(Mil|C)]elp( tr" f acts( Mi\|C)). (7.21) 


q'€clp—'(q) 
Note the similarity with the definition of the projection of a probabilistic execution fragment 
(cf. Section 4.3.2). 
The probabilistic execution H satisfies the following properties. 


a. H is a probabilistic execution of M;||C. 


The fact that each state of H is reachable can be shown by a simple inductive argument; 
the fact that each state of H is a finite execution fragment of M;||C follows from a simple 
analysis of the definition of clip. 


From (7.21) it is enough to check that for each closed state gq’ of H’, the transition 
elp( try" | acts(.M;||C)) is generated by a combination of transitions of M;||C. Since tr 


is a transition of H’, (ir | acts(M;||C)) can be expressed as 7; pj(q/~ tr;), where each 
tr; is a transition of M;||C’. We distinguish three cases. 
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1. tr; is a non-shared transition of M;. 
Then tr; = ((s,c),a,P © D(c)) for some action a and probability space P, where 
(s,c) = Istate(q’). Let Istate(clp(q')) = (s’,c'). Then, s’ = s, as it follows directly 
from the definition of clp. Define tri to be the transition ((s,c’),a4,P@ D(c’)). Then 
tr’, is a transition of M;||C and elp(q’~ tr;) = elp(q')~ tr’ 

2. tr; is a non-shared transition of C’. 
Then tr; = ((s,c),a,DP(s)® P) for some action a and probability space P, where 
(s,c) = Istate(q'). Let Istate(clp(q')) = (s’,c’). Then, s’ = s and c’ = ¢, as it follows 
directly from the definition of clp after observing that q’ must be a closed state in 
order to enable tr;. Define tr’ to be tr;. Then tr’ is a transition of M;,||C and 
elp(q! ~ trj) = elp(q!) > r5 

3. tr; is a shared transition. 
Then tr; = ((5, Ctr), a, P © D(ct,)) for some action a and probability space P, where 
(s, cy) = lstate(q’). In particular, c;, is one of the states that are added to those 
of C, and tr is a simple transition of C with action a. Moreover, from each state 
(s',cir) © Qpep(e,), there is a transition ((s', ct), @2,D(s’) ® Pi) enabled. Let 
Istate(clp(q')) = (s', ce"). Then, s’ = s. Define tr’, to be ((s,c’),a,P @ Pry). Then, 
from the definition of C’, tri, is a transition of Mj||C. 


Observe that clp distributes over combination of transitions. Moreover, from Equa- 
tion (7.19), observe that for each j elp(q' ~ trj;) = elp(q')~ tri. Thus, elp( tr" t 
acts( M;||C)) = elp(q')~ (32, pjtrs), which is generated by a combination of transitions of 
M,C. 


. For each state q of H, 


Py[C,| = » Pr[Cq']. (7.22) 


q’Emin(elp—1(q)) 


This is shown by induction on the length of g. If ¢ consists of a start state only, then the re- 
sult is trivial. Otherwise, from the definition of the probability of a cone, Equation (7.21), 
and a simple algebraic simplification, 


Pr[Cyas] = Pa[Cq] ( » a) ’ (7.23) 


q'€elp—*(q) 


where f(qas) expresses the probability of the completions of q’ to a state whose col- 
lapse gives gas without using actions from V, U V2 in the first transition. Formally, 
if a is not a shared action, then F(qas) is PH a x elp~'(qas)]; otherwise, F,(qas) 
is PH (a, da(s', €or) Ps cy, )l(d2, Cals’, cer )aa(s’,€))], where c;, = Istate(q')[C’, and 
s = (s’,c). In the first case, Qi M ({a} x elp~'(qas)) contains only one element, say 
(a,q’'as”), and Pr [Cy|Fy(qas) gives Px [Cas]; in the second case Pr [Cy] Fy (gas) 
gives Pr [C( 


q'a(s',ctr)ags)]- 
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Observe that the states of min(clp~'(qas)) are the states of the form described above 
(simple cases analysis). Thus, by applying induction to (7.23), using (7.20), simplifying 
algebraically, and using the observations above, 


Pr[Coas] = S- Py [Cy’]. (7.24) 
gq’! Emin(clp—1 (gqas)) 
c. tdistr(H) = tdistr(H') [ acts(.M;||C). 


Let @ be a finite trace of H or H’. Then {a € Qy | 8 < trace(a) f acts(M;||C)} can be 
expressed as a union of disjoint cones Use@C, where, if the last action of @ is a and a is 
not a shared action, 


0 = {¢ € states( H') | trace(q) | acts(M;||C) = B, lact(q) = a}, (7.25) 
and if the last action of @ is a and a is a shared action, 

0 = {¢ € states( H’) | trace(q) | acts(M;||C) = G, lact(q) = az}. (7.26) 
Observe that © is a set of closed states. The set clp(Q) is the set 

clp(O) = {gq € states( H) | trace(q) = f, lact(q) = a}, (7.27) 


which is a characterization of {a € Qy | B < trace(a)} as a union of disjoint cones. 
Observe that min(elp~'(clp(@))) = ©. Moreover, for each q # qo of elp(®), elp~'(q1) N 
clp-'(q2) = 9. Thus, from (7.22), Py [UgeoCy] = Pr[Ugecip(o)Cy|. This is enough to 
conclude. 


To complete the proof of (7.14) it is enough to observe that Hy = clp( Hj). Property (7.14) is 
then expressed by property (c). 

Suppose by contradiction that it is possible to obtain D’ from M2||C’. Consider the scheduler 
that leads to D’ in M3||C’, and let H5 be the corresponding probabilistic execution. First, we 
build a new probabilistic execution H4 of M2||C’ whose trace distribution is D’, and such that 
each shared action a is followed immediately by its corresponding action a2. Then we let H2 be 
clp( H4'). This leads to a contradiction since tdistr( Hz) = D. The rest of the proof is dedicated 
to the construction of HY. 

For each state q of H4, let exch(q) be the set of sequences q’ that can be obtained from q¢ 
as follows: each sequence 


(50, Cir )@( $1, Ctr )T2( $2, Ctr) ++ * Ta Sn, Ctr )42( Sh; ©) 
is replaced with 

(80, Ctr )A( $1, Cty )42( 51, €)T2(S2,€) +++ Ta( Sp, ©), 
each sequence 


(so, Ctr )a( 51, Ctr )T2( 82, Ctr) vee Th( Sh, Cir) 
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occurring at the end of ¢ is replaced with 
(so, Ctr a( 51, Ctr )42(S1, €)T2(S2, c) _— Th( Sh; c), 


where c is any of the states that a2 may lead to from c,,, and each sequence 


(So, Ctr )a( $1, Ctr) 


occurring at the end of g, where a is a shared action, either it is replaced with 


(so, Ctr )a( 51, Ctr )d2(S1, c), 
where c is any of the states that a2 may lead to from c;,, or it is not replaced. Then, define 


states(H!/) = LJ each(q). (7.28) 
q€states( Hs) 


Let (¢,P) be a restricted transition of #5, and suppose that no action of V2 occurs. Let q’ be 
a state of exch(q) that does not end with a shared action. Then, for each (a,q) € 9 there is 
exactly one g, € exch(q,) such that q’ < qj, and |q,| = |q’| + 1 (simple analysis of the definition 
of exch). Denote such gj by exchy(q1). Let 2’ = {(a, exchy(q1) | (a, q1) € Q}, and let, for each 
(a,q,) € 2’, P’[(a, q)] = Pl(a x exch—'(qi))], where exch—'(q) is the set of states q’ of HS such 
that ¢ € exch(q'). Then define the transition exch,/((q,P)) to be 


Aa 


erchy((q,P)) = (q',P’). (7.29) 


For each state q of H4, let min(exch—'(q)) be the set of minimal states of exch—'(q) under 
prefix ordering. For each state q’ of exch™'(q), where q is closed, let 


e Pin 2 Puy [Cy] if q’ is closed, i.e., if each occurrence of a shared action a is followed 
eventually by an occurrence of its corresponding action a9; 


q 4 


e py = PH [Cy] Pile] if 7 is open, where Istate(q’)[C’ = cy and Istate(q)[C = ¢. 


For each q’ € exch~'(q), let 


q 
_erch—1(q) A Py 


Py! (7.30) 


Do g!'€min(exch! (¢)) Pin , 


If the last action of g is a shared action a, and Istate(q) = (s, cr), then the transition enabled 
from q in Hf is 


qo ((8, Ctr), 2, D(s) ® Pir). (7.31) 
If the last action of g is not a shared action, then the transition enabled from gq in H4 is 


Soper) pl lacts(H4)\Veleacha (tri? t (acts(H})\V2)). (7.32) 


q 
q’€exch—(q) 


The probabilistic execution AH satisfies the following properties. 
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a. HJ is a probabilistic execution of M||C’. 


The fact that each state of H4/ is reachable can be shown by a simple inductive argument; 
the fact that each state of HJ is a finite execution fragment of Mg ||C’ follows from a 
simple analysis of the definition of exch. 

We need to check that for each state q of HY the transition enabled from gq in H% is 
generated by a combination of transitions of M2||C’. If the last action of ¢ is a shared 
action, then the result follows immediately from Expression (7.31) and the definition of 
C’. If the last action of ¢g is not a shared action, then consider a state q’ € exch—'(q). 
A; 


The transition tr,,? | (acts(H3)\V2) can be expressed as )°; pi(q'~ tri), where each tr; is 


a transition of M2||C’ enabled from Istate(q’). We distinguish three cases. 


1. tr; is a non-shared transition of Mo. 

Then tr; = ((s,c),a,P @ P(c)) for some action a and probability space P, where 
? ? ? Pp y Pp ? 

(s,c) = Istate(q'). Let Istate(q) = (s',c’). Then, s’ = s. Define tr’ to be the 
transition ((s,c’),a,P@D(c’)). Then tr’ is a transition of Ma||C’ and exch,(q'~tr;) = 
q@ tr’. 

2. tr; is a non-shared transition of C’. 
Then tr; = ((s,c),a,P(s) ® P) for some action a and probability space P, where 
(s,c) = Istate(q’). Let Istate(q) = (s',c’). Then, s’ = s and c = c’. Define tr’ to be 
tr;. Then tr! is a transition of Mg||C’ and exch,(q'~ tr;) = q7 tri. 

3. tr; is a shared transition. 


Then tr; = ((s,c),a,P © D(c)) for some action a and probability space P, where 
(s,c) = Istate(q'). Let Istate(q) = (s',c’). Then, s’ = s and c = c’. Define tr‘ to be 
tr;. Then tri is a transition of M2||C’ and exch,(q'~ tr;) = ¢7 tr}. 
Observe that exch distributes over combination of transitions. Thus, exch,((trg:) f 
(acts( H5)\V2)) can be expressed as >>, p;(q~ tr’), which is generated by a combination of 


transitions of M||C’. From (7.32), the transition enabled from q in H% is generated by a 
combination of transitions of M||C’. 


b. For each state g of HY, 


Do g!Emin(exch—1(q)) Pu: [Cy] if g ends with a shared action, 


q . 
Do g!Emin(exch—1(q)) Py otherwise. 


PaylC,] = (7.33) 


The proof is by induction on the length of g. If q consists of a start state only, then the 
result is trivial. Otherwise, consider Puy (Cas: We distinguish two cases. 
1. g is open. 
In this case, since in H4 each shared action is followed immediately by the corre- 
sponding action of V2, a is an action of V2. Moreover, from the definition of exch, 


exch~'(q) = min(exch—'(qas)) = min(exch™'(q)), (7.34) 
and all the elements of exch—'(q) are open states. From induction, 


Pry |Cq] = > Pr [Cq)- (7.35) 


q’Emin(exch—"(q)) 
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Let ¢ = s[Mo, and let cj, = Istate(q)[C’. Then, for each gq’ € min(exch'(q)), 
tp = lstate(q')[C", and 
pit? = Py (Cy Pel (7.36) 


i 


Moreover, Pye [(a, gas)] = Py,[c]. Thus, from the definition of the probability of a 
cone and (7.35), 


Pry [Coas] = S- Pr [Cq\ Perle]: (7.37) 
gq’ Emin(exch—"(q)) 


By using the fact that min(exch—'(q)) = min(exch~'(qas)), and using (7.36), we 
obtain 
Pr [Cqas] = S- pe. (7.38) 
q/€min( exch (qas)) 
2. q is closed. 
In this case, from the definition of the probability of a cone and (7.32), 


_exch—1 HH; _ 
PulCool= Paste [ DP Pax exch Mons) (7.39) 


q’€exch—1(q) 


Let Ptr,[q'| denote P;,[c], where c = Istate(q)[C’, and cy, = Istate(q’)[C’. Then, 
from induction and (7.30), 


HS = 
Pyu(Cqas) = S- Py [Cy] P,1’ [a x exch "(qas)] + (7.40) 
q’€erch—1(q)|closed(q’) 
HS = 
» Pas{Cq|Ptrgla| Py? [a x each (qas)]. 


q'€exch—*(q)|open(q’) 

We distinguish two subcases. 

(a) ais a shared action. 
In this case each state q’ of exch~'(q) such that Pe [a x exch'(qas)] > 0 
is closed. Thus, only the first summand of (7.40) is used. Moreover, each 
state of min(exch™*(qas)) is captured by Expression (7.40). Thus, Py [Cqas] = 
yg'Emin( exch" (qas)) PH [Cy]. Observe that gas is open. 

(b) a is not a shared action. 
In this case, for each q/ € exch—'(q), if q’ is closed, then all the states reached in 
QA ({a} x exch! (qas)) are closed, and if q' is open, then all the states reached 
in QM ({a} x exch '(qas)) are open. Moreover, each state of min(exch~'(qas)) 
is captured by Expression (7.40). Thus, from the definition of Pin Pus [Coas] = 
Doy!emin(erch—!(qas)) Pq! Observe that gas is closed. 


c. tdistr( H$) = tdistr( H%). 


Let @ be a finite trace of H} or Hy. Then {a € Qy: | B < trace(a)} can be expressed as 
a union of disjoint cones UzeoC, where 


0 = {q € states( H"’) | trace(q) = , lact(q) = lact(3)}. (7.41) 
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We distinguish two cases. 


1. @ does not end with an action of V3. 

The set 0’ = {gq € exch(O) | lact(q) = lact(9)} is a characterization of {a € Qyzy | 
B < trace(a)} as a union of disjoint cones. Observe that min(erch—'(@’)) = © and 
that for each pair of states q, 4 qo of O', min(exch!(q)) M min(exch!(qz)) = 
0. Thus, if 6 ends with a shared action, then (7.33) is sufficient to conclude that 
Py[{a € Oy | 8 < trace(a)}] = Pyy[{a € Qyy | 8 < trace(a)}]; if B does not 
end with a shared action, then, since all the states of O are closed, Equation (7.33) 
together with the definition of Pin are sufficient to conclude. 


2. 2 ends with an action of V2. 


In this case § = §’az for some action a2 € Vz. Observe that, both in H4 and HY, 
after the occurrence of a shared action a the corresponding action a2 occurs with 
probability 1: for H4 recall that tdistr( 4) | acts(M2||C) = D; for HJ see (7.31). 
Thus, the probability of @ is the same as the probability of 3’, and the problem is 
reduced to Case 1. | 


Lemma 7.5.4 Let C be a distinguishing separated context for two probabilistic automata My 
and Mz. Then there exists a distinguishing cycle-free separated context C’ for My and Mg. 


Proof. C’ can be built by unfolding C. Every scheduler for M;||C can be transformed into a 
scheduler for M;||C’ and vice versa, leading to the same trace distributions. a 


Lemma 7.5.5 Let C be a distinguishing cycle-free, separated context for two probabilistic au- 
tomata M, and M2. Then there exists a distinguishing cycle-free separated context C’ for My 
and My, with a transition relation that is at most countably branching. 


Proof. Let D be a trace distribution of M,||C that is not a trace distribution of Mg||C. 
Consider the corresponding probabilistic execution H. Observe that H has at most countably 
many states, and that at each state of H there are at most countably many transitions of C 
that are scheduled. Thus, in total, only countably many transitions of C’ are used to generate 
P. Then C’ is C without the unused transitions. | 


Lemma 7.5.6 Let C be a distinguishing cycle-free, separated context for two probabilistic au- 
tomata My, and Mz such that the transition relation of C is at most countably branching. Then 
there exists a distinguishing cycle-free separated context C’ for My, and Mp» that at each state 
either enables two deterministic transitions or a unique probabilistic transition with two possible 
outcomes. C" is called a binary separated context. 


Proof. For each state s of C, choose a new action start,. Let s enable the transitions 
try, trg,..., where each tr; is a transition (s,a;,P;). The transition relation of C’ is obtained in 
two phases. First, a transition is chosen nondeterministically as shown in Figure 7-7, where each 
symbol e denotes a distinct state and each symbol 7 denotes a distinct internal action; then, for 
each state e;, the transition tr; is encoded as follows. Let 0; be {s;1, 5;,2,..-}, pij 2 Pi[sij]; 


and Dj,j = »e>; Pik» The transition relation from e; is represented in Figure 7-8, where each 


154 


Figure 7-7: Nondeterministic choice of a transition. 


Si Sia Si3 
e e e 
tT t A t . 
Pia Pi2/ Pir Pi3/ Pi3 
Pir Pi3/ Piz 


@, T @ q Oeste eS 


Figure 7-8: Transforming a transition into binary transitions. 


symbol e denotes a distinct state and each symbol 7 denotes a distinct internal action. Observe 
that by scheduling all the transitions of the diagram above, for each 7 we have 


Piss) =F ses) (7.42) 


where P[s;;] is the probability of reaching s;; from e;. Denote the set of actions of the kind 
start, by Vstarz- Denote the auxiliary actions of C’ that occur between a start action and a 
state e; by Vj, and denote the auxiliary actions of C’ that occur between a state e; and the 
corresponding occurrence of action a; by V2. 

Let D be a trace distribution of Mj||C that is not a trace distribution of M2||C’. Consider 
a probabilistic execution H, of M,||C whose trace distribution is D in M,||C’, and consider the 
scheduler that leads to Hy in M,||C. Apply to Mj||C’ the same scheduler with the following 
modification: whenever some transition of C’ is scheduled, schedule the start action from C’, 
then schedule the internal transitions to choose the transition of C’ to perform with the right 
probability, and then schedule the transitions of the chosen transition till the corresponding 
external action of C occurs. Denote the resulting probabilistic execution by Hj and the resulting 
trace distribution by D’. Then, 


D' | acts(M,||C) = D. (7.43) 


To prove (7.43), we define a new construction, called shrink and abbreviated with shr, to be 
applied to probabilistic executions of M;||C’ such that no action of M; occurs between a state 
of the form e; and the occurrence of the corresponding action a; of C, and such that all the 
transitions between a state of the kind e; and the corresponding occurrences of action a; are 
scheduled. 

Let H' be such a probabilistic execution of M;||C’. Denote shr(H’) by H. A state q of H’ 
is closed if each occurrence of a state of the kind e; is followed eventually by the occurrence of 
the corresponding action a;. For each state g of H’ let shr(q) be obtained from q as follows: 
each sequence 


(So, €o) Start, (So, @)b1(51, @) > + +b; (Sp, ©; )T1( Sn, ©) ++ Th(Sn, @)a;(S, €) 
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is replaced with 

(S0, €0)b;, (Si, , Co) ++ + Bi, (Si, Co) a; (5, €), 
where 7t,,..., 2 is the ordered sequence of the indexes of the b’s that are actions of M;, and each 
sequence either of the form 

(So, €o) start, (So, @)b1(51, @) ++ -b;(Sp, ©; )T1( Sn, ©) °° Th(Sn,@) 


or of the form 


(so, Co) start, (So, @)b1(51, @) ---bn(Sp, ©) 


occurring at the end of ¢ is replaced with 
(So, €0)b;, (Si, , Co) ++ + Bi, (Si,1, Co); 

where 21,...,%; is the ordered sequence of the indexes of the 6’s that are actions of M;. Then, 
states(H) = {shr(q)|q € states(H’)}. (7.44) 


Let (q,P) be a restricted transition of H’, and suppose that no action of acts(C’)\acts(C) 
occurs. Let 2’ = {(a,shr(q’)) | (a,¢q’/) € OQ}, and for each (a,q”) € 0’, let P’[(a,¢q")] = 
Pla x shr~*(q')], where shr~*(q) is the set of states q/ of H’ such that shr(q’) = q. Then the 
transition shr((q,P)) is defined to be 


shr((q,P)) = (shr(q),P). (7.45) 


For the transition relation of H, consider a state q of H, and let min(shr—'(q)) be the set of 
minimal states of shr~'(q) under prefix ordering. For each state ¢ € shr—'(q), let 


_shr—(q) Pi [Cal 
a a (7.46) 
" Yogemin(shr-1(q)) PHC] 


The transition enabled from g in A is 
S> ops PH [acts( Mj||C)] shr(trl’ f acts(M;||C)). (7.47) 
q’€shr—"(q) 
The probabilistic execution H satisfies the following properties. 


a. H is a probabilistic execution of M;||C. 


The fact that each state of H is reachable can be shown by a simple inductive argument; 
the fact that each state of H is a finite execution fragment of M;||C follows from a simple 
analysis of the definition of shr. 


We need to show that for each state q of H the transition of Expression (7.47) is generated 
by a combination of transitions of M;||C. The states of shr—'(q) that enable some action 
of M;||C can be partitioned into two sets ©, and ©, of closed and open states, respectively. 


We analyze ©, first. Let q/ € ©,.. Since try is a transition of H’, (try | acts(M;||C)) can 
be expressed as )>; p;(q’~ tr;), where each tr; is a transition of M;||C’. We distinguish 
two cases. 
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l. tr; is a transition of M;. 
Then tr; = ((s,c),a,P © D(c)) for some action a and probability space P, where 
(s,c) = Istate(q’). Let Istate(shr(q')) = (s’,c’). Then, s’ = s, as it follows directly 
from the definition of shr. Moreover, (s,a,P) is a transition of M;. Define ir’ 
to be the transition ((s,c’),a,P @ D(c’)). Then tr’ is a transition of M;||C and 
shr(q'~ tri) = q7 tr’. 

2. tr; is a transition of C’. 
This case is not possible since, from the construction of C’, no action of C can be 
enabled from a closed state. 


Observe that shr distributes over combination of transitions. Thus, 


shr(try - acts(.M;\|C)) = LaPil (shr(q')~ tr’), (7.48) 


which is generated by a combination of transitions of M;(||C. 


We now turn to ©,. The set ©, can be partitioned into sets (O;);s0, where each set 
©; consists of those states q’ of O, where a particular state e; of C’ occurs without its 
matching action a;. Each element q’ of ©; can be split into two parts q ~ q@2, where 
Istate(q,)[C’ = e;. Denote q by head(q’). Partition ©; into other sets (O;%)k>0, where 
each O;, is an equivalence class of the relation that relates two states iff they have the 
same head. Denote the common head of the states of 0; by head(O;,;). For each pair 
of states q4,q2 of H’ such that q < q, denote by p,,,, the probability value such that 
Pul(CE] = Pal(Cy 


4, Pag: Then, for each equivalence class @;,;, the expression 


S> pl PH Tacts(Mi|C)|shr(tr’ | acts(My||C)) (7.49) 


qg 
VEO; x 


can be rewritten into 


—shr 
(mis, » Phead(q | 
- ‘ECO; k 


Phead(q’)q’ 


Forces Paccagng tt Malshe(tral t acts(Mil\C)) (7.50) 
VEO; q jk ea 


where (7.50) is obtained from (7.49) by expressing each par) as Pee Phew (a gt, by 


ua for each q¢’ os O;,;, by substituting pH [a;] 


for PH Lacts(Mi|\C) (action a; is the only action of M;||C that can be performed from q’ 
due to the structure of H’), and by multiplying and dividing by gee, , Phead(q')q! 


grouping Pes? a) “)? which is equal to pe na (q! 


Observe that each transition that appears in (7.50) is generated by some transitions of 
M,||C. Thus, the transition of (7.50) is generated by a combined transition of M;||C. 
Denote this transition by tr;,. Then, in Expression (7.47) it is possible to substi- 


tute each subexpression )°,co, Da ‘a PI [acts( Mil|C )]shr(trg | acts(M;||C)) with 


h . 
(Dye ad? Ly'eo, , Phead(q')q’)t? j,k- This is enough to conclude. 
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b. For each state g of A, 


Py[C,] = S- Py [Cy’]. (7.51) 


gq! Emin(shr—*(q)) 


This is shown by induction on the length of g. If q consists of a start state only, then the 
result is trivial. Otherwise, from the definition of the probability of a cone and (7.47), 


PulCosl= YD PlCy\PH"[ax shr-™(qas)) (7.52) 
q/Eshr—*(q) 


Observe that the states of min(shr—'(qas)) are the states that appear in (ax shr7'(qas))N 


Q, for some g! € shr“!(q). Thus, Pr[Cgas] = Dg'emin(shr-'(qas)) PH Cal 


c. tdistr(H) = tdistr(H') [ acts(.M;||C). 


Let 2 be a finite trace of H or the projection of a finite trace of H’. Then {a € Qy | 8B < 
trace(a) | acts(M;||C)} can be expressed as a union of disjoint cones U;eoCy where 


0 = {q € states(H') | trace(q) } acts(M;||C) = G, lact(q) = lact(3)}. (7.53) 
Observe that © is a set of closed states. The set shr(O) is the set 
shr(O) = {q € states(H) | trace(q) = B, lact(q) = lact(B)}, (7.54) 


which is a characterization of {a € Qy | B < trace(a)} as a union of disjoint cones. 
Observe that min(shr7*(shr(®))) = ©. Moreover, for each q # q of shr(@), shr7!(q)N 
shr—'(q2) = 0. Thus, from (7.51), Pr[UgcoCy] = Pula € shr(O)Cy]. 


To complete the proof of (7.43), it is enough to observe that Hy = shr( Hj). Property (7.43) is 
then expressed by property (c). 

Suppose by contradiction that it is possible to obtain D’ from M2||C’. Consider the scheduler 
that leads to D’ in M3||C’, and let H5 be the corresponding probabilistic execution. First, we 
build a new probabilistic execution HY of M2||C’ whose trace distribution is D’, such that there 
is no action of My between each state of the kind e; and the occurrence of the corresponding 
external action of C’, and such that all the transitions between a state of the kind e; and the 
corresponding occurrences of action a; are scheduled. Then we let Hz = shr(H%). This leads 
to a contradiction since tdistr(H2) = D. The rest of the proof is dedicated to the construction 
of HY. 

For each state g of H4, let shf(q) be the set of sequences q’ that can be obtained from q as 
follows: each sequence 


(80, @; )b1(s1,¢) ++ -be( se, @)a;(s, €) 
is replaced with 


(80, ©; )bi, (So, ©) +++ bi, ($0, ©) @j(S0, €)B ky (Sky ©) ++ Din (8, ©) 


158 


where 71,...,% is the ordered sequence of the indexes of the b’s that are actions of C’, and 
ky,...,m is the ordered sequence of the indexes of the 6’s that are actions of M2; each sequence 


(so, ©; )b1(51, ) .. bi (Sk, e) 
occurring at the end of ¢ either is replaced with 
(50, ©; )bi,(S0,@) ++ -8i,(S0,¢) 7 a (S0, ©)4;(S0, €)Bi, (Sk 56) ++ Phim (S56) 


where 71,..., 7%; is the ordered sequence of the indexes of the 6’s that are actions of C’, ky,..., km 
is the ordered sequence of the indexes of the 6’s that are actions of M2, and a, called an extension 
for q, is an arbitrary execution fragment of Mo||C’ that leads to the occurrence of aj, or, is 
replaced with a prefix of (59, ¢;)b;,(so,)---b;,(so,¢). Then, 


states(H#) = LJ shf(q). (7.55) 
q€states(H5) 


Let (q, P) be a restricted transition of H$, and suppose that only actions of Mz and Vetar¢ occur. 
Let q' be astate of shf(q). Then, for each (a, q1) € 9 there is exactly one qi € shf(q.) such that 
q <q and |q| = |q'| +1. Denote such q by shfy(qi). Let Q' = {(a, shfy(q) | (a,a) € OQ}, 
and let, for each (a, q,) € 2’, P’[(a, q)] = Pl(ax shf~'(q,))], where shf—'(q) is the set of states 
q of H4 such that q € shf(q'). Then define the transition shf,.((q,P)) to be 


shf w((q,P)) = (q',P). (7.56) 


For each state q of Hi, let min(shf—'(q)) be the set of minimal states of shf—'(q) under prefix 
ordering. Let q be a closed state of H¥, and let q’ € shf—'(q). If q’ is an open state, then let a 
be the extension for q’ that is used in q, and let EX, be the product of the probabilities of the 


edges of a. For each state q of shf~'(q), where q is closed, let 


° pi = Pr [C4] if q' is closed; 


e Par s Pry [Cy EX, if q’ is open. 
For each q’ € shf—'(q), let 


q 
shf-1(q) A ee ee (7 57) 
= 7: . 
do g!'Emin(shf—*(q)) Pa’! 


If ¢ is open, then the transition enabled from q in H4 is the one due to the transition of C’ 
enabled from Istate(q)[C’; if ¢ is closed, then the transition enabled from q in His 


op PR? facts HS)\(acts(C) U Va)] (7.58) 
q/€shf—*(q) 


shf (tr? } (acts( H4)\(acts(C) U V9))). 


The probabilistic execution HJ satisfies the following properties. 
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a. HJ is a probabilistic execution of M||C’. 

The fact that each state of H4 is reachable can be shown by a simple inductive argument; 
the fact that each state of HY is a finite execution fragment of M2||C’ follows from a 
simple analysis of the definition of shf. 

We need to check that for each state q of HY the transition enabled from gq in H% is 
generated by a combination of transitions of M2||C’. If q is an open state, then the result 
follows immediately from the definition of the transition relation of H4’. If q is a closed 
state, then consider a state q’ € shf—'(q). The transition wre? | (acts(H$)\V2), which 
appears in Expression (7.58), can be expressed as >>, p;(q’ ~ tr;), where each tr; is a 
transition of M2||C’ enabled from Istate(q’). We distinguish two cases. 


1. tr; is a transition of Mo. 
Then tr; = ((s,c),a,P © D(c)) for some action a and probability space P, where 
(s,c) = Istate(q’'). Let Istate(q) = (s',c’). Then, s’ = s. Define tr’ to be the 
transition ((s,c'),a,P@D(c')). Then tr} is a transition of M||C’ and shf ,(q’~ tr;) = 
q@ tr’. 

2. tr; is a transition of C’. 
Then tr; = ((s,c),a,P(s) ® P) for some action a and probability space P, where 
(s,c) = Istate(q'). Let Istate(q) = (s',c'). Then, s’ = s and c = c’ (q is closed). 
Define tr; to be tr;. Then tri is a transition of M2||C’ and shf,(q'~ tri) = q7 tri. 


Observe that shf distributes over combination of transitions, and thus, the transition 
shf (tr | (acts( H)\V2)) can be expressed as >>; p;(q~ t;), which is generated by a 
combination of transitions of M||C’. 


b. For each state g of HY, 


_ Do glemin(shf—(q)) Py! if g is closed, 
Pru [Cy] = yo g'emin(shf—2(q)) PH|Ca'] if g is open. (7.59) 


The proof is by induction on the length of g. If q consists of a start state only, then the 
result is trivial. Otherwise, consider Pry [Cqas)- We distinguish two cases. 


1. g is open. 
In this case a is an action of V2 U acts(C), and each state of shf~'(q) is open. From 
the definition of the probability of a cone and induction, 


Hi! 
Pyn[Cqas] = ( S- Pry ] P,? [(a, gas)]. (7.60) 
q’Emin(shf—"(q)) 

We distinguish two other cases. 

(a) acée V2. 
Observe that all the states of min(shf—'(q)) enable the same transition of C’ 
that is enabled from g. Moreover, for each q/ € min(shf—'(q)), action a occurs 
with probability 1 (in D’ each occurrence of a start action is followed by an 
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external action with probability 1), and the probability of reaching a state of 


min(shf—'(qas)) given that a occurs is P24, qas)] (recall that q enables only 
action a). Since all the states of min(shf~'(qas)) are open and have a prefix in 
min(shf—'(q)), we can conclude 
Pry (Cqas] = » Pr [Cq].- (7.61) 
q/Emin(shf—*(qas)) 
(b) a € acts(C). 
From the definition of HY, P14, qas)| = 1. Observe that all the states of 
min(shf—'(q)) enable the same transition of C that is enabled from ¢. Moreover, 
for each gq’ € min(shf~'(q)), action a occurs with probability 1 (in D’ each 
occurrence of a start action is followed by an external action with probability 
1), leading to a state of shf—'(qas) for sure (recall that q enables only action a). 
Thus, for each q’ € shf—'(q), 
PuslCyl = 3 PaslCyrl: (7.62) 
q!Emin(shf—* (qas))|q’<a” 
Combining (7.60) and (7.62), we obtain 
Pry [Cqas] = » Pr [Cq].- (7.63) 
q/Emin(shf—*(qas)) 
For each q’ € min(shf~'(qas)), if 7 is open, then pi = = Py: [Cy] by definition; 
if q’ is closed, then pi;* = Pr [Cy] since EA = 1 (no a must be added by shf 
to get q' from qas). Thus, (7.63) becomes 
Put [Cqas] = 3 ee (7.64) 
g/€min(shf—1(gas)) 
2. q is closed. 
In this case, from the definition of the probability of a cone and (7.58), 


_shf—1 HS _ 
PylCqas) = Pry(Ca] Sp OPI la x sh i) (7.65) 
q'€shf—*(q) 
From induction, the definition of pl), and an algebraic simplification, 
HAS _ 
Palas] = S| PnlCylP lax shf"(qas)]+ (7.66) 
q/€shf—1(q)|closed(q') 
S- Pr [Cy EYP) Tax shf—'(qas)]. 


q'€shf—'(q)|open(q') 
We distinguish two subcases. 
(a) gas is open. 
In this case each state q’ of shf—'(q) such that pie [a x shf~'(qas)] > 0 is 
closed, and thus only the first summand of (7.66) is used. Moreover, for each gq’ 
of shf—'(q) the set Qn? Na x shf—'(qas) is made of open states q/as’ such that 
Eaiyst! = 1. Observe that all the states of min(shf—'(qas)) are captured. Thus, 
Pru (Cqas] = > pi. (7.67) 
a! €min(shf— (gas) 
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(b) gas is closed. 

In this case, for each q' € shf—'(q), if q’ is closed, then all the states reached in 
QM ({a} x shf~*(qas)) are closed, and if q' is open, then all the states reached 
in Q4 7 ({a} x shf~*(qas)) are open and the extension a does not change, i.e., 
the term EF does not change. Observe that all the states of min(shf~'(qas)) are 
captured. Thus, 

Put [Cqas] = S- pi. (7.68) 

g/€min(shf—1(gas)) 


c. tdistr( H$) = tdistr( H4). 


Let 3 be a finite trace of Hj or HY. Then {a € Qyy | 9 < trace(a)} can be expressed as 
a union of disjoint cones UgeoC,. We distinguish two cases. 


1. 8 does not end with an action of C. 
Then 


0 = {q € states( H"’) | trace(q) = , lact(q) = lact(3)}. (7.69) 


The set 0’ = {g¢ € shf(O) | lact(q) = lact(3)} is a characterization of {a € Oy» | 
B < trace(a)} as a union of disjoint cones. Observe that min(shf—'(@’)) = © and 
that for each q # q2 of O', min(shf~'(q1))M min(shf~'(qo)) = 9. Thus, from (7.51), 
Pyrlta € Qa |B < trace(a)j] = Pyy[ta € Quy | 9 < trace(a)}]. 
2. 2 ends with an action of C. 

In this case 6 = (’a; for some action a; € acts(C). Since in H5 and HY after the 
occurrence of a state e; the corresponding action a; occurs with probability 1, we 
can assume that all the states of © end in e,, i.e., 


© = {q € states( H’) | trace(q) = 9’, and Istate(q) is one of the e;’s}. (7.70) 


Then the set 0’ = min(shf(O)) is a characterization of {a € Qyy |B < trace(a)} asa 
union of disjoint cones. Observe that all the elements of © are open. Property (7.59) 
is sufficient to conclude. | 


Lemma 7.5.7 Let C be a distinguishing binary separated context for two probabilistic automata 
My, and Mg. Then there exists a distinguishing total binary separated context C’ for M, and 
Mp where all the probabilistic transitions have a uniform distribution. C"' is called a balanced 
separated context. 


Proof. We achieve the result in two steps. First we decompose a binary probabilistic transition 
into several binary uniform probabilistic transitions, leading to a new distinguishing context 
C1; then we use Lemma 7.5.4 to make C into a cycle-free context. 

The context C, is obtained from C’ by expressing each probabilistic transition of C by 
means of, possibly infinitely many, binary probabilistic transitions. For each state s of C, let 
start, be a new action. If s enables a probabilistic transition with actions a1, a to states 51, 59, 
respectively, and with probabilities p,, po, respectively, then Cy enables from s a deterministic 
transition with action start,. Then, Cy, enables an internal probabilistic transition with a 
uniform distribution. If py > p2 (po > pi), then one of the states that is reached enables a 
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deterministic transition with action a, (a2). The other state enables a new internal probabilistic 
transition with a uniform binary distribution, and the transitions from the successive states are 
determined by giving a, probability 2(p, — 1/2) and ag probability 2p2 (a, probability 2p; 
and a2 probability 2(p2 — 1/2)). If p1 = po, then one state enables a,, and the other state 
enables aj. For example, if py = 5/8 and pz = 3/8, then the corresponding transitions of Cy 
are represented below. Let D be a trace distribution of M,||C that is not a trace distribution 


fe 


2 51 
ay ag ay 
t t t 
1/2 1/2 
Starts 1/2. 1/2 1/2 
>@ 7 @ 7 @ 7 = @ ager 


of M2||C. Consider a probabilistic execution H, of M,||C whose trace distribution is D, and 
consider the scheduler that leads to Hy in M,||C. Apply to M,||C, the same scheduler with 
the following modification: whenever a probabilistic transition of C' is scheduled, schedule the 
start action from Cj ,, then schedule the internal transitions to resolve the probabilistic choice, 
and finally schedule the chosen action. Denote the resulting probabilistic execution by H{ and 
the resulting trace distribution by D’. Then, 


S 


D' | acts(M,||C) = D. (7.71) 


To prove (7.71), we define a new construction shr;, similar to shr, to be applied to probabilistic 
executions of M;||Cy such that no action of M; occurs between the occurrence of a start, action 
and the occurrence of one of the corresponding external actions of C’, and such that all the 
transitions of C between the occurrence of an action start, and the occurrence of one of the 
corresponding external actions of C are scheduled. The new function is identical to shr if we 
consider each state reached immediately after the occurrence of a start action like the states e; 
used in Lemma 7.5.6. We leave the details to the reader. 

Suppose by contradiction that it is possible to obtain D’ from M9||C,. Consider the scheduler 
that leads to D’ in M2||Cy, and let H{ be the corresponding probabilistic execution. First, we 
build a new probabilistic execution Hy of Mo||C, whose trace distribution is D’, such that 
no action of MM; occurs between the occurrence of a start, action and the occurrence of one 
of the corresponding external action of C’, and such that all the transitions of C, between 
the occurrence of an action start, and the occurrence of one of the corresponding external 
action of C' are scheduled. Then we let Hz = shri(H4/). This leads to a contradiction since 
tdistr( Hz) = D. 

The construction of H4’, which is left to the reader, is the same as shf if we consider each 
state reached immediately after the occurrence of a start action like the states e; used in 
Lemma 7.5.6. a 


Lemma 7.5.8 Let C’ be a distinguishing balanced separated context for two probabilistic au- 
tomata My, and Mz. Then there exists a distinguishing binary separated context C’ for My, 
and My with no internal actions and such that each action appears exactly in one edge of the 
transition tree. C’ is called a total balanced separated context. 
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Proof. The context C’ is obtained from C by renaming all of its actions so that each edge of 
the new transition relation has its own action. 

Let D be a trace distribution of M,||C that is not a trace distribution of M2||C. Consider a 
probabilistic execution H, of M,||C whose trace distribution is D, and consider the scheduler 
that leads to Hy in M,||C. Apply to M,||C’ the same scheduler with the following modification: 
whenever a transition of C' is scheduled, schedule the corresponding transition of C’. Denote the 
resulting probabilistic execution by Hj and the corresponding trace distribution by D’. From 
construction, HM, and Hj are the same up to the names of the actions of C. Thus, if p’ is the 
function that maps each action of C’ to its original name in C', D = p'(D’) (the renaming of a 
trace distribution is the probability space induced by the function that renames traces). 

Suppose by contradiction that it is possible to obtain D’ from M2||C’. Consider the scheduler 
that leads to D’ in Ma||C", and let H4 be the corresponding probabilistic execution. Apply to 
Mz2\|C the same scheduler with the following modifications: whenever a transition of C’ is 
scheduled, schedule the corresponding transition of C’ with the unrenamed actions. Let H2 be 
the resulting probabilistic execution. From the construction, Hz and H} are the same up to 
the names of the actions of C. Thus, tdistr( Hz) = p'(D’) = D, which is a contradiction. = 


Lemma 7.5.9 Let C be a distinguishing total balanced separated context for two probabilistic 
automata M, and M2. Then there exists a distinguishing total balanced separated context C"’ 
for M, and Mz that from every state enables two deterministic transitions and a probabilistic 
transition with a uniform distribution over two choices. C’ is called a complete context. 


Proof. In this case it is enough to complete C’ by adding all the missing transitions and states. 
If D is a trace distribution of M;||C' that is not a trace distribution of M2||C,, then it is enough 
to use on My||C’ the same scheduler that is used in M,||C. In fact, since each new transition 
of C’ has a distinct action, none of the new transitions of C’ can be used in M||C’ to generate 
D. = 


Lemma 7.5.10 Let C be a distinguishing complete context for two probabilistic automata My 
and Mz. Then the principal context Cp is a distinguishing context for M, and Mo. 


Proof. The result is achieved in two steps. First the actions of C are renamed so that each state 
enables two deterministic transitions with actions left and right, respectively, and a probabilistic 
transition with actions pleft and pright. Call this context Cy. Then, by observing that each 
state s of Cy is uniquely determined by the trace of the unique execution of C, that leads to s, 
all the states of C, are collapsed into a unique one. 

Thus, we need to show only that C is a distinguishing context. Let D be a trace distribution 
of M,||C that is not a trace distribution of M2||C’. Consider the scheduler that leads to D in 
M,\|C, and apply to M,||C; the same scheduler with the following modification: whenever a 
transition of C’ is scheduled, schedule the corresponding transition of C;. Denote the resulting 
trace distribution by D’. Note that if we rename all the actions of Cj into their original name 
in C, then we obtain D. 

Suppose by contradiction that it is possible to obtain D’ from Mg||C,. Consider the sched- 
uler that leads to D’ in M ||Cy, and apply to Mg||C the same scheduler with the following 
modification: whenever a transition of Cy is scheduled, schedule the corresponding transition 
of C’. The resulting trace distribution is D, which is a contradiction. | 
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Lemma 7.5.11 Let Cp be a distinguishing context for two probabilistic automata My and My). 
Then the simple principal context, denoted by C, is a distinguishing context for My and Mo. 


Proof. Let D be a trace distribution of M,||Cp that is not a trace distribution of M ||Cp. 
Consider a probabilistic execution Hy of M,||Cp whose trace distribution is D, and consider 
the scheduler that leads to Hy in M,||Cp. Apply to M,||C the same scheduler with the follow- 
ing modification: whenever the probabilistic transition of Cp is scheduled, schedule the start 
action of C’ followed by the next transition of C’ that becomes enabled. Denote the resulting 
probabilistic execution by Hj and the resulting trace distribution by D’. Then, 


D' | acts(M,|\|Cp) = D. (7.72) 


To prove (7.72), we define a new construction shrz, similar to shr, to be applied to probabilistic 
executions of M;||C such that no action of M; occurs between the occurrence of a start action 
and the occurrence of one of the actions pleft and pright, and such that the transitions labeled 
with pleft and pright occur whenever they are enabled. The new function is identical to shr 
if we consider each state reached after an action start as a state of the kind e;. We leave the 
details to the reader. 

Suppose by contradiction that it is possible to obtain D’ from M2||C’. Consider the scheduler 
that leads to D’ in Mg||C’, and let H{ be the corresponding probabilistic execution. First, we 
build a new probabilistic execution Hy of M||C whose trace distribution is D’, such that no 
action of My, occurs between the occurrence of a start action and the occurrence of one of 
the actions pleft and pright, and such that the transitions labeled with pleft and pright occur 
whenever they are enabled. Then we let Hz = clp,(H4). This leads to a contradiction since 
tdistr( Hz) = D. 

The construction of H4’, which is left to the reader, is the same as shf if we consider each 
state reached immediately after the occurrence of a start action like the states e; used in 
Lemma 7.5.6. = 


Proof of Theorem 7.5.1. Let My Epc M2. Then, from Lemma 7.5.11, Mi||Cp Cp Mg||Cp. 
Conversely, let My||Cp Cp M2\||Cp. Then, from Lemmas 7.5.3, 7.5.4, 7.5.5, 7.5.6, 7.5.7, 7.5.8, 
7.5.9, and 7.5.10, M, Epc Mo. 7 


7.6 Discussion 


A trace-based semantics similar to ours is studied for generative processes by Jou and Smolka 
[JS90]. One of the processes of Jou and Smolka is essentially one of our probabilistic executions. 
The semantics of a process is given by a function, called a trace function, that associates a prob- 
ability with each finite trace. Since our trace distributions are determined by the probabilities 
of the cones, our trace distributions are characterized completely by the trace functions of Jou 
and Smolka. In other words, the trace semantics of Jou and Smolka is the semantics that we 
use to say that two probabilistic executions have the same trace distribution. 

Jou and Smolka define also a notion of a maximal trace function. Given a probabilistic 
execution H, the interpretation of a maximal trace function in our framework is a function that 
associates with each finite trace @ the probability of the extended executions on Qy that end in 
6 and whose trace is 3. Jou and Smolka show that the trace function of a process is sufficient 
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to determine the maximal trace function of the process. In our trace distributions the maximal 
trace function of a probabilistic execution is given by the probability of each finite trace in the 
corresponding trace distribution. From the definition of a trace distribution the probability of 


each finite trace is determined uniquely by the probabilities of the cones, and thus the result of 
Jou and Smolka holds also in our framework. 
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Chapter 8 


Hierarchical Verication 
Simulations 


8.1 Introduction 


In Chapter 7 we have studied the trace distribution precongruence as an instance of the hierar- 
chical method for the verification of probabilistic systems. Another instance of the hierarchical 
method is called the simulation method. According to the simulation method, rather than 
comparing two probabilistic automata through some abstract observations, two probabilistic 
automata are compared by establishing some relation between their states and by showing that 
the two probabilistic automata can simulate each other via the given relation. Standard work 
on simulation relations appears in [Mil89, Jon91, LV91]. Simulation relations are stronger than 
the trace preorder, and are often used as a sound proof technique for the trace preorder. 

In this chapter we study how to extend some of the relations of [Mil89, Jon91, LV91] to the 
probabilistic framework. We start with the generalization of the simplest relations that do not 
abstract from internal computation, and we conclude with the generalization of the forward 
simulations of [LV91] that approximate closely the trace distribution preorder. We prove the 
equivalent of the Execution Correspondence Lemma [GSSL94] for probabilistic automata, which 
states that there is a strong connection between the probabilistic executions of two probabilistic 
automata related by some simulation relation. Finally, we use the new Execution Correspon- 
dence Lemma to prove that the existence of a probabilistic forward simulation is sufficient to 
prove the trace distribution precongruence relation. 


8.2 Strong Simulations 


One of the finest equivalence relations for ordinary automata would be graph isomorphism; 
however, it is widely recognized that graph isomorphism distinguishes too much. A coarser 
equivalence relation is strong bisimulation [Par81, Mil89], where two automata A; and Ag are 
equivalent iff there is an equivalence relation between their states so that for each pair (51, s2) 
of equivalent states, 


. a . . a 
if s; —> s/, then there exists a state s4 equivalent to si such that sy —> s}. 
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Figure 8-1: The difference between strong bisimulation and the kernel of strong simulation. 


That is, Ay and Ag simulate each other. A preorder relation that is closely connected to 
strong bisimulation is strong simulation. An automaton A, is strongly simulated by another 
automaton Ag» iff there is a relation between the states of A; and the states of A» so that for 
each pair (51,52) of related states, 


* a * a * 
if s; —> s{, then there exists a state s4 such that s2 —> s4 and s{ is related to 8%. 


The kernel of strong simulation is an equivalence relation that is coarser than bisimulation. 


Example 8.2.1 (Strong simulation and strong bisimulation) Figure 8-1 shows the dif- 
ference between strong bisimulation and the kernel of strong simulation. The double-arrow 
dotted links represent a strong bisimulation between A; and Ag; thus, A, and A» are strongly 
bisimilar. There is also a strong simulation from Ag to As, expressed by the dotted lines that 
have an arrow pointing to A3, and a strong simulation from A3 to Aj, expressed by the dotted 
lines that have an arrow pointing to Ag. Thus, Ag and A: are equivalent according to the kernel 
of strong simulation. However, there is no bisimulation between Az and Ag since state 5 of As 
must be related to state s; of Ay in order for Ay to be able to simulate the transition s9 —> s9 


of Ag, but then it is not possible to simulate the transition s1 ars 83 of Ap from sy in A3. 


The extension of strong bisimulation and strong simulation to the probabilistic framework 
presents a problem due to the fact that a probabilistic transition leads to a probability distri- 
bution over states rather than to a single state. Thus, a relation over states needs to be lifted 
to distributions over states. Here we borrow an idea from [JL91]. 

Let RC X x Y bea relation between two sets X,Y, and let P, and P2 be two probability 
spaces of Probs(X) and Probs(Y), respectively. Then P, and P2 are in relation Cr, written 
P, Er Po, iff there exists a weight function w: X x Y — [0,1] such that 


1. foreach t € X, Vyey w(2,y) = Pilz], 
2. for each y EY, Doex w(e,y) = Pall 


3. for each (7,y) EX x Y, if w(a,y) > O then # R y. 


Example 8.2.2 (Lifting of one relation) The idea behind the definition of Cr is that each 
state of 2; must be represented by some states of Q2, and similarly, each state of Q2 must 
represent one or more states of 2). Figure 8-2 gives an example of two probability spaces that 
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Figure 8-2: Lifting one relation. 


are related. The dotted lines connect states that are related by R. Thus, state s; can be 
represented by s2, for a third of its probability, and by s2. for the reminder. Similarly, state 
S2,9 represents 8; for one sixth of its probability and s, 2 for the reminder. A useful property 
of ER is its preservation over combination of probability spaces. | 


If R is an equivalence relation, then we denote the relation Ep alternatively by =r. The reason 
for the alternative notation is that whenever FR is an equivalence relation and Py, =r P2, each 
equivalence class of # is assigned the same probability in P, and P2 (cf. Lemma 8.2.2). 

The definition of strong bisimulation and strong simulation for probabilistic automata are 
now straightforward. For convenience assume that M, and M2 do not have common states. 
A strong bisimulation between two simple probabilistic automata M,, M2 is an equivalence 
relation R over states(M,)U states( M2) such that 


1. each start state of Mj, is related to at least one start state of M2, and vice versa; 


2. for each pair of states s; R s2 and each transition s; —~+ P, of either M, or Mo, there 
exists a transition s2 —~ P of either M, or Mz such that Py =p P2. 


We write My ~ M2 whenever acts(M,) = acts(M2) and there is a strong bisimulation between 
M, and Mo. 

A strong simulation between two simple probabilistic automata M,, Mo is a relation RC 
states(M,) x states(Mz) such that 


1. each start state of Mj, is related to at least one start state of Mo; 


2. for each pair of states s; R s9 and each transition s} —+ P, of M1, there exists a transition 
83 —> P2 of Mz such that Py CR Pr. 


We write M, Css Mz whenever acts(M,) = acts( M2) and there is a strong simulation from My 
to Mz. We denote he kernel of strong simulation by =ss. Because of Lemma 8.2.2, our strong 
bisimulations are the same as the bisimulations of [Han94], and our strong simulations are a 
generalization of the simulations of [JL91]. 

It is easy to check that ~ is an equivalence relation, that Egg is a preorder relation, and 
that both ~ and Egg are preserved by the parallel composition operator. 

We conclude this section by proving two results about the lifting of a relation. The first 
result shows that the lifting of a relation is preserved by the combination of probability spaces; 
the second result shows that P, =r P2 iff P, and P2 assign the same probability to each 
equivalence class of R. 
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Lemma 8.2.1 Let Px; Er Py; via a weight function w;, and let {Piti>o be a family of 
real numbers between 0 and 1 such that Y°;.5 pi) = 1. Then > o;59 piP xi ER Djso piPy,i via 
Dido PiWi- 


Proof. Let Px = )ijs0 PiPx,i, Py = Uiso PiPy,i, and w = dijo piwi. Let x € Qy. Then 


Dyeay V(2,Y) = Cycay Viso PiVAe,Y) = Liso Pi(Xycay wil, y)) = Viso viPx iz] = Px[z].- 
Condition 2 of the definition of CR is verified similarly. For Condition 3, let w(2,y) > 0. Then 
there exists an 7 such that wi(a,y) > 0, and thus a R y. a 


Lemma 8.2.2 Let X,Y be two disjoint sets, R be an equivalence relation on X UY, and let 
P, and P2 be probability spaces of Probs(X) and Probs(Y), respectively. Then, Py =r P2 iff 
for each equivalence class C of (X UY)/R, Py[CO AQ] = P2[CN Qg]. 


Proof. Suppose that Py =r P2, and let w be the corresponding weight function. Then, for 
each equivalence class C' of (X UY)/R, 


P[CnN QQ] = S- Pi[z] = S- S- w(2,y), (8.1) 


rECnQy, rECnQy, yECNQ2 


and 


PICNM%)= SY) Plyl= dS dO w(2,y). (8.2) 


yECNQ2 yECNQ2 rECnQy, 


From the commutativity and associativity of sum, 
PCN] = Pf Qg]. (8.3) 


Conversely, suppose that each equivalence class (X UY )/R has the same probability in P, and 
P2. We define w(x, y) for each equivalence class of (X UY)/R, and we assume implicitly that 
w is 0 for all the pairs (7,y) € Q| x Q2 that are not considered in the construction below. 
Consider any equivalence class C of (X UY )/R, and let ¥'=CNQ, and Y'=CNQz2. From 
hypothesis we know that P,[X"] = P2[Y’]. Let x1, 22,... be an enumeration of the points of 
X’, and let y,y2,... be an enumeration of the points of Y’. For each 7, let pj = ype; Pilxil 
and let qi = >>p<; Poly]. Then 


w(2;.4;) = 0 if pi4a < Gj OF Gian < Di 
ead min(pi41, Gj+1) — max(p;,gq;) otherwise. 


Informally, the construction above works as follows. Consider two intervals [0, Pi[X‘]], and 
mark the first interval with the points p; and the second interval with the points q;. Each 
interval [p;,pi41] has length P,[z;] and each interval [¢;,¢;41] has length P[y;]. The weight 
function w(2;,y;) is defined to be the length of the intersection of the intervals associated with 
x; and y;, respectively. It is simple to verify that w is a weight function for Py, and P2. | 


170 


Figure 8-3: Combining transitions to simulate a transition. 


8.3. Strong Probabilistic Simulations 


In the definition of strong bisimulations and strong simulations we have not taken into account 
the fact that the nondeterminism can be resolved by combining several transitions probabilis- 
tically into a unique one. That is, a transition of a probabilistic automaton could be simulated 
by combining several transitions of another probabilistic automaton. 


Example 8.3.1 (Combining transitions to simulate another transition) Consider the 
two probabilistic automata M, and M2 of Figure 8-3. Mz contains the transitions of M, plus 
a transitions that is obtained by combining probabilistically the transitions of M,. For this 
reason there is no simulation from Mz to M, (the additional transition cannot be simulated). 
On the other hand, M,; and M2 have exactly the same probabilistic executions, and therefore 
we do not see any reason to distinguish them. | 


Example 8.3.1 suggests two new relations, which are coarser than strong bisimulation and strong 
simulation, where the only difference is that a transition can be simulated by a probabilistic 
combination of transitions. 

For convenience assume that M4, and M>) do not have common states. A strong probabilistic 
bisimulation between two simple probabilistic automata M,, M2 is an equivalence relation R 
over states(M,) U states( M2) such that 


1. each start state of Mj, is related to at least one start state of M2, and vice versa; 


2. for each pair of states s; R s2 and each transition s; —+ P, of either M, or Mo, there 
exists a combined transition s2 —+c¢ P2 of either M, or Mz such that Py =p Po. 


We write M, ~p M2 whenever acts(M,) = acts( M2) and there is a strong probabilistic bisim- 
ulation between My and Mo. 

A strong probabilistic simulation between two simple probabilistic automata MM, and Mp is 
a relation RC states(M,) x states( M2) such that 


1. each start state of Mj, is related to at least one start state of Mo; 


2. for each pair of states s; R s9 and each transition s; —+ P, of M,, there exists a combined 
transition sy +c Pz of Mz such that Py Lr Po. 


We write M,; Csps M2 whenever acts(M,) = acts( Mz) and there is a strong probabilistic 
simulation from M, to Mj. We denote the kernel of strong probabilistic simulation by =sps. 
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It is easy to check that ~p is an equivalence relation, that Cgps is a preorder relation, and 
that both ~p and Cgpg are preserved by the parallel composition operator. It is easy as well 
to verify that a strong bisimulation is also a strong probabilistic bisimulation and that a strong 
simulation is also a strong probabilistic simulation. 


8.4 Weak Probabilistic Simulations 


The abstraction from internal computation can be obtained in the same way as for ordinary 
automata: a transition of a probabilistic automaton should be simulated by a collection of 
internal and external transitions of another probabilistic automaton. For the formal definition 
we use the weak combined transitions of Chapter 4. 

For convenience assume that M4, and Mz do not have common states. A weak probabilistic 
bisimulation between two simple probabilistic automata M4, and Mp is an equivalence relation 
R over states( M1) U states( M2) such that 


1. each start state of Mj, is related to at least one start state of M2, and vice versa; 


2. for each pair of states s; R s2 and each transition s; —+ P, of either M, or Mo, there 
afext(M2) 


exists a weak combined transition s. = qc P2 of either My, or M2 such that Py =r Po. 
We write M, =p M2 whenever ext(M,) = ext( M2) and there is a weak probabilistic bisimulation 
between M, and Mo. 
A weak probabilistic simulation between two simple probabilistic automata M, and Mo is a 
relation RC states(M,) x states( M2) such that 


1. each start state of Mj, is related to at least one start state of Mo; 


2. for each pair of states s; R s2 and each transition s; —+ P, of Mj, there exists a weak 


. oye M: = 
combined transition s2 at eather) Pz of Mz such that P, Er P2. 


We write M,; Cwps M2 whenever ext(M,) = ext( M2) and there is a weak probabilistic simula- 
tion from M, to Mz. We denote the kernel of weak probabilistic simulation by =wps. 

It is easy to verify that a strong probabilistic bisimulation is also a weak probabilistic 
bisimulation and that a strong probabilistic simulation is also a weak probabilistic simulation. 
However, it is not as easy to verify that =p is an equivalence relation, that LCwps is a preorder 
relation, and that both =p and Cwps are preserved by the parallel composition operator. The 
verification of these properties is a simplification of the verification of the same properties for 
the relation of the next section. For this reason we omit the proofs from this section. 


8.5 Probabilistic Forward Simulations 


One of the main results of this chapter is that all the relations presented so far are sound for 
the trace distribution precongruence. However, none of the relations of the previous sections 
allow for one probabilistic operation to be implemented by several probabilistic operations. 
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Figure 8-5: A more sophisticated implementation. 


Example 8.5.1 (Weak probabilistic simulations are too coarse) Consider the two prob- 
abilistic automata of Figure 8-4. The probabilistic automaton M2, which chooses internally one 
element out of four with probability 1/4 each, is implemented by the probabilistic automaton 
M,, which flips two fair coins to make the same choice. However, the first transition of MM, 
cannot be simulated by Mp. since the probabilistic choice of Mp2 is not resolved completely yet 
in M,. This situation suggests a new preorder relation where a state of M, can be related 
to a probability distribution over states of My. The informal idea behind a relation s; R P2 
is that s; represents an intermediate stage of M4, in reaching the distribution 2. For exam- 
ple, in Figure 8-4 state s; would be related to a uniform distribution P over states s4 and sj, 
(P = U(s5, s/,)), meaning that s1 is an intermediate stage of My in reaching the distribution P. 

It is also possible to create examples where the relationship between s and P does not mean 
simply that s is an intermediate stage of M4, in reaching the distribution P, but rather that 
& is an intermediate stage in reaching a probability distribution that can be reached from P. 
Consider the two probabilistic automata of Figure 8-5. Although not evident at the moment, 
M, and Mp) are in the trace distribution precongruence relation, i.e., My Epc M2. Following 
the same idea as for the example of Figure 8-4, state s, is related to U(s5, 5). However, s1 is 
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not an intermediate stage of M, in reaching U/(s, s/,), since s; enables a transition labeled with 
an external action 1, while in My no external action occurs before reaching U(s5, 84). Rather, 
from s5 and s there are two transitions labeled with /, and thus the only way to simulate 


the transition s; —> U(s3, 84) from U(s5, s/,) is to perform the two transitions labeled with 
1, which lead to the distribution U(s+, 8%, 84, 549). Now the question is the following: in what 
sense does U/(s4, sg, 34, 84) represent U/(s3,s4)? The first observation is that sz can be seen as 
an intermediate stage in reaching U/(s¢, sg), and that s4 can be seen as an intermediate stage in 
reaching U(s9, 549). Thus, sz is related to U(st, sg) and sq is related to U(s, 549). The second 
observation is that U(s+, 8%, 56, $49) can be expressed as 1/2U(s+, 58) + 1/2U(s$, 349). Thus, 
U( s+, 8%, 85, 319) can be seen as a combination of two probability spaces, each one representing 
an element of U/(s3, 4). This recalls the lifting of a relation that we introduced at the beginning 
of this chapter. | 


Based on Example 8.5.1, we can move to the formal definition of a probabilistic forward simu- 
lation. A probabilistic forward simulation between two simple probabilistic automata M, and 
Mz is a relation RC states(M,) x Probs(states(M2)) such that 


1. each start state of Ad, is related to at least one Dirac distribution over a start state of 
Mo; 
2. for each s R P’, if s “+ Py, then 


M. 
(a) for each s’ € 2’ there exists a probability space P, such that s’ ah ext) Ps, and 


(b) there exists a probability space P3 of Probs( Probs(states( M2))) satisfying P; Cr P5, 


such that So yeq P’[s’|Py = Pen, PS[PIP. 


We write My Crg Mz whenever ext(M,) = ext( M2) and there is a probabilistic forward simu- 
lation from M4, to Mo. 


Example 8.5.2 (A probabilistic forward simulation) The probabilistic forward simula- 
tion for the probabilistic automata M, and My. of Figure 8-5 is the following: so is related 
to U(so); each state s;, 1 > 7, is related to D(s‘); each state s;, 1 < i < 6, is related to 
U(s5:41, 8:42). It is an easy exercise to check that this relation is a probabilistic forward 
simulation. Observe also that there is no probabilistic forward simulation from Mz to My. In- 
formally, s4 cannot be simulated by M,, since the only candidate state to be related to s{ is 51, 
and s; does not contain all the information contained in s4. The formal way to see that there 
is no probabilistic forward simulation from My) to My, is to observe that Mz and M, are not in 
the trace distribution precongruence relation and then use the fact that probabilistic forward 
simulations are sound for the trace distribution precongruence relation (cf. Section 8.7). In 
Mg\|Cp it is possible force action left to be scheduled exactly when M2 is in s4, and thus it 
is possible to create a correlation between action left and actions a and 6; in M,||Cp such a 
correlation cannot be created since action left must be scheduled before action J. | 


It is easy to check that a weak probabilistic simulation is a special case of a probabilistic forward 
simulation where each state of MM, is related to a Dirac distribution. The verification that Crs 


174 


is a preorder that is preserved by parallel composition is more complicated. In this section 
we show that Lys is preserved by parallel composition; the proof that Lys is a preorder is 
postponed to Section 8.6.4. 


Proposition 8.5.1 Crs is preserved by the parallel composition operator. 


Proof. Let My Crs Mo, and let R be a probabilistic forward simulation from My, to M3. Let 
R' be a relation between states(.M,) x states( M3) and Probs(states( M2) x states(.M3)), defined 
as follows: 


(51,83) R’ P iff P = Pp © D(s3) for some P2 such that s1 R Po. (8.4) 


Condition 1 of the definition of a probabilistic forward simulation is immediate to verify. Con- 
dition 2 for transitions that involve M, only or M3 only is immediate to verify as well. 

Let (1,53) R’ Py ® D(s3), and let (51,83) + P; © P3, where s; + Pi, and s3 —> 
P3. From the definition of a probabilistic forward simulation, for each s € Q2 there exists 
a weak combined transition s; —+c¢ P, of M2, and there exists a probability space P) of 
Probs( Probs( states( M2))), such that 


SS” Po[s]Ps = SY PSPIP, (8.5) 


sEQ2 PEM, 


and 


P, Er P3. (8.6) 


For each s € Q2, let O, be a generator for s —S«q P,. Define a new generator Of as follows: 
for each finite execution fragment a of M2||Ms starting in (s, 53), 


1. if O,(a[ Mz) = (s’,P), where (s',P) = 3°; p;(s', a;, P;), each (s',a;, P;) is a transition of 
My, and a[ Mz = s3, then 


O%(a) = do Pil(s', 83), Gi, Pi @ P), 


4 


where 
P! = D(sa) if a; # a, and Pl = Py if a; = a. 


2. if O,(a[M2) = (s',P), where (s’,P) = ¥°; p(s’, a;, P;), each (s’,a;, P;) is a transition of 
Mo, a[ M3 = s3a85, and s4 € Qs, then 


O7(a) = do pil(s’ 83), aj, Pi @ D(s3)); 


4 


3. if none of the above cases holds, then Of(a) = D(6). 
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The weak combined transition generated by each O% is (s, 83) —>+c P,;@P3. In fact, an execution 
fragment a of M2||Mz is terminal for Of iff a[ Mz is terminal for Os and a[M3 = s3as4 for 


83 € Q3, and thus Qe, = O, x Q3. Moreover, for each a € Qo:, Py? = = PM, P3{Istate(a|M3)}. 
Denote P, ® P3 by P(s,s,)- Then, for each (s, 3) € Q2 x Pls), we have identified a weak 


combined transition (s, 3) —>c P(s,s3)- These are the spaces of Condition 2.a in the definition 
can be expressed alternatively as 


of a probabilistic forward simulation. Note that P(s,53) 
P(s,s3) = > P3[s°] (Ps ® D(s83)) . (8.7) 
shEQ3 
Let 
Py3 = So Palss] (P} @ D(D(s5))), (8.8) 
8,EQ3 


where the pairing of two probability spaces is meant to be their product. For each s4 € Qs, 
since Py Er P35, Pi © P(s3) Er Py @ D(D(s3)). Thus, from Lemma 8.2.1, Pi @ P3 Er P33. 
This is enough to show that Condition 2.b of the definition of a probabilistic forward simulation 
is satisfied. 


We are left with )),co, Pa[s]P(s,s3) = Leen, P3,3[P]P, which is shown as follows. From (8.7), 


S- Pp[s|P( (s,83) =S S- PIs [s] P3[s3] (Ps ® D(s3)) . (8.9) 


sEQ2 sEQ2 sh EQ3 

From (8.5), 
Y= Pls]P(ss2) = SY) >> P3[P]Ps[s5] (P @ D(ss)) . (8.10) 
bEQD> s,€Q3 PEM, 


From a simple algebraic manipulation, 


d— Pals]P(s,s) = do » Ps[ss)P3[P]P. (8.11) 


sEQ2 $,€Q3 PEI! ap(D(sh)) 

From (8.8), 
Pals (o.80) = S- PS 4[P]P. (8.12) 
bEQD> PEQ 


8.6 The Execution Correspondence Theorem 


The existence of some simulation relation between two probabilistic automata implies that there 
is some strict relation between their probabilistic executions. This relationship is known as the 
execution correspondence lemma for ordinary automata [GSSL94] and is useful in the context 
of liveness. In this section we prove the execution correspondence theorem for probabilistic 
automata; a corollary, which is proved in Section 8.7, is that the existence of a probabilistic 
forward simulation is sound for the trace distribution precongruence. 
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Figure 8-6: Fringes. 


8.6.1 Fringes 


Let H be a probabilistic execution of a probabilistic automaton M. Define the extended states 
of H, denoted by extstates(H), to be states(H)U {¢6 | ¢ € states(H), Py[Cys] > 0}. A fringe 
of H is a discrete probability space P of Probs(extstates( H)) such that for each state q of H, 


Ss” Pla] < PalCy). (8.13) 
gEQla<aq' 


Two fringes P, and P2 are in the < relation iff for each state q of A, 


Ss. Bld SS Pole’), (8.14) 


qEQ4 |<! q’EQ2 |g<q! 


Informally, a fringe is a line that cuts a probabilistic execution in two parts (see Figure 8-6). A 
fringe is smaller than another one if the first fringe cuts the probabilistic execution earlier than 
the second fringe. Figure 8-6 shows three fringes Ff, Fy and F3, where Fy < Fh < F3. 

A fringe of particular interest is the fringe that cuts a probabilistic execution fragment at 
some depth %. Let fringe( H,7) denote the fringe of H where 2 = {q © extstates(H) | |q| = 
t}U{¢6 © extstates( H) | |q| < i}, and for each q € Q, Plq] = Px[Cy]- 


8.6.2. Execution Correspondence Structure 


Let R be a probabilistic forward simulation from My, to Mz. An execution correspondence 
structure via R is a tuple (H1, H2,m,5), where Hy is a probabilistic execution of My, H2 is a 
probabilistic execution of Mj, m is a mapping from natural numbers to fringes of M2, and S$ 
is a mapping from natural numbers to probability distributions of Probs( Probs(states( H2))), 
such that 


1. For each 7, m(2) < m(i + 1); 
2. For each state gg of He, lim; eins P;|q| = Px[Ca; 
3. Let qm RP iff for each q € 2, trace(q) = trace(q,), and either 


(a) q does not end in 6, each state of 2 does not end in 6, and Istate(q,) R Istate(P), 
or 


(b) q@ and each state of 2 end in 6 and Istate(6-strip(q,)) R Istate(6-strip(P)). 


Then, for each 7 > 0, m(t) = PENG Psqy[P]P, and fringe( Ay, t) Cr S(i). 
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Figure 8-7: Execution Correspondence Structures: the role of Condition 2. 


& 


4, Let, for each ¢ > 0, each qi € fringe(H1,7), and each qz € states(H2), Wi(m, q2) 
Sp wila, P)Plq]. If Wila,¢5) = 0 for each prefix or extension 5 of q2, then, for each 
extension g of g such that gj) € fringe(H,,i +1) and each prefix or extension gq) of qo, 
Wisi(G %) = 9. 


Informally, an execution correspondence structure is an object that shows how a probabilistic 
execution Hy of M, is represented by a probabilistic execution Hy of Mz via R. Ho is said to 
be the probabilistic execution fragment that corresponds to H;. Conditions 1 and 3 state that 
each fringe fringe(H1,7) is represented by the fringe m(7) in Hz, and Condition 2 states that 
at the limit each state of H2 represents some part of H,. Figure 8-7 gives an example of an 
execution correspondence structure (left) and of a structure that fails to satisfy Condition 2 
since state g is not captured (right). Condition 4 enforces the correspondence between Hy and 
Hy. Informally, it states that if two states gq, and gg of H, and Ho, respectively, are connected 
through the i‘ fringes, then for each j < i there are two prefixes gi and qi of q and q, 
respectively, that are connected through the j*" fringes. This condition allows us to derive a 
correspondence structure between the execution fragments of M, and Mp that denote the states 
of H, and Hz. We do not use Condition 4 to prove any of the results that we present in this 
thesis; however, this condition is necessary to prove the results that Segala and Lynch present 
in [SL94]. 

If # is a weak probabilistic simulation, then an execution correspondence structure is a 
triplet (H1, H2,m): Condition 3 becomes fringe(H1,71) Cr m(i), where q R qo iff trace(q) = 
trace(q2) and either q, and q end in 6 and 6-strip(lstate(q,)) R 6-strip(lstate(qz)), or lstate(q,) R 
Istate(q2); Wilqi,q2) becomes w;(qi1,9q2), and Condition 4 says that for each 7 > 0, given 
qm € fringe(H1,7) and q © states( Hz), if wi(m,¢q) = 0 for each prefix or extension gq) of qa, 
then, for each extension q of q such that qj € fringe(H,,i+ 1), and each prefix or extension 
gq Of G2, Wi41(G1,G) = 0. 
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If R is a strong probabilistic simulation, then an execution correspondence structure is a pair 
(H,, Hz): Conditions 1 and 2 are removed; Condition 3 becomes fringe(H1,7) Cr fringe( H2,%) 
where q R q2 iff itrace(q,) = ttrace(q2) and either q and q: end in 6 and 6-strip(Istate(q,)) R 
6-strip(istate(qz)), or Istate(q,) R Istate(qz2); Condition 4 says that for each 7 > 0, given q@ € 
fringe( Hy, 7) and q2 € fringe( Ho, 7), if wi(qi,q2) = 0, then, for each extension qi of q such that 
q, € fringe( H,i+1) and each extension gq of gz such that qf € fringe( H2,i+1), wi4i(G, @) = 0. 


8.6.3 The Main Theorem 


Theorem 8.6.1 Let M, Ers M2 via the probabilistic forward simulation R, and let Hy be a 
probabilistic execution of M,. Then there exists a probabilistic execution Hz of Mz, a map- 
ping m from natural numbers to fringes of Mz, and a mapping S from natural numbers to 
probability distributions of Probs( Probs(states( Hz))), such that (Hy, H2,m,5') is an execution 
correspondence structure via R. 


Proof. Let q, be a state of H,, and let P, be a distribution over potential states of H2 such 
that q, Er P2 according to the definition given in the definition of an execution correspondence 
structure. Denote by PH, the probability space such that tnd = Viren Pi, [tr](q 7 tr). Let 


try € On,» and let P:,, be the probability space reached in q, ~ try. 

Since # is a probabilistic forward simulation, then for each state gq. of Q2 there exists a 
weak transition trg,p,tr,q, of Hz with action a [ ext( Mz), leading to a distribution over states 
PoPotriqg, Such that there exists a probability distribution over probability distributions of 
potential states of H2, denoted by PP? psi satisfying 


S- Pe stim [P|P = S- P2[q2\Pa1 Po tryqo (8.15) 
PENS ptr, q2EQ2 
and 
Per, CR PP Potry (8.16) 


via a weight function wy, p,1r,- Denote the probability space °c, P2ld2]Pa:P2triq. bY Pa, Poin» 
i.e., 


A 
Po,Potrs = S- P2[q2]Po,Potriqe- (8.17) 
92 EQ2 


Denote the generator of each weak transition trg,p,1r,q. DY Og, Potryqo (cf. Section 4.2.7). For the 
sake of this proof, we change the notation for the generators of the transitions of a probabilistic 


execution. Thus, for each q5 such that q2 < ¢, Og, Po tr, q.(qh) stands for Og, 7, tr: 92(G2 192), and 
pouPatria stands for pouPatria 
q 94142 . 
For each state q and each probability distribution over states Po, let 6,, = P(q4), dp, s 
A 


Yarer, Palaa]é.,, 67, = D(dp,), and wsq,p, be a weight function such that ws,p,(q16, P2) = 1. 
Note that, if for each q2 € Qe, trace(q1) = trace(qz), then 


by, Cr 8p, (8.18) 
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Via W§q,P.. Moreover, 


dp, = Pss [PIP. (8.19) 
PEQs, 


Let sy be the start state of H,, and sg be a start state of M2 that is related to s,;. We know 
that s2 exists since R is a probabilistic forward simulation. Let Active be the smallest set such 
that 


1. (51, D(s2)) € Active; 


2. if (1, P2) € Active, try € OF, and (q,,P2) € Qtr x 8 Dyin then (q),P3) € Active; 


3. if (q@1,P2) € Active, Pj} [6] > 0, then (q16, 53) € Active. 


Observe that for each pair (q1,P2) € Active, q R P2 (simple inductive argument). For each q 
such that there exists some P2 with (q1,P2) € Active, each try € Qn,» and each gz € Qe, let 
active(q, P2, tri, qz) be the set of states that are active in Oy, p,1r,q,, and let reach(q, P2, tri, q2) 
be the set of states that are reachable in Q,,p,%r,9,- Let active denote the union of the sets 
reach(q1, P2, tr1,q2) where (q1,P2) € Active, tri € Qn,» and gg € Q2. For each 2 < 0, let 
Active(i) be the set of pairs (q1,P2) € Active such that either |q:| = 7% or |qi| < ¢ and q ends 
in 6. For each pair (qq, P2) of Active such that q does not end in 4, let 


Aa 


Py = D> Pi ltriPin + PH [6] 8 (8.20) 


a 
try eQp, 


be the probability space reached in H, with the transition enabled from q,, 


Aa 


PoP. = yy Pi, [tri\ParPotr1 + PH, [6]ép, (8.21) 


try Ene, 
be the probability space that is reached in the corresponding transition of P2, 


S 4 S S 
Pars = > Pi, [tri]Po Pots + Pi, [6]63, (8.22) 
try Ene, 
be the probability space of probability spaces that corresponds to P,,, and for each gj, P3, 
Aa 


Wa, P2(G15 P2) = yy PH, [tra}WarPotrr (qi; P2) + PH, [4] wsq:P2(415 Po) (8.23) 


a 
try eQp, 


be the corresponding weight function. From Lemma 8.2.1, 


Pa Cr Pap, (8.24) 


via the weight function wy,,p,. 
For each pair (q1,P2) of Active such that q ends in 6, let 


Aa 


= D(P2), and Wq,P2(M1, Pa) 


Py, = Din), Pup, = Pr, PF = 


Sp, 1. (8.25) 


It is immediate to observe that Equation (8.24) holds also in this case. 
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Define m(i), S(2) and w; inductively as follows. 


m(0) = D(s2), S(0) = D(m(0)),  wols1,m(0)) = 1, (8.26) 

mi+t1) = S- wild, P2)PaP os (8.27) 
(q1,P2)€ Active (2) 

SG+1) = S- wi(q,P2)P? p,. (8.28) 
(q1,P2)€ Active (2) 

wigi(G, Ps) = S>wilgr, P2)WarP2( 41, PS). (8.29) 


(q1,P2)€ Active (2) 


To show that Equations (8.27), (8.28),and (8.29) are well defined, we show by induction that 
for each t > 0, do (a, Pa)eActive(i) Wi(G1» P2) = 1. The base case is a direct consequence of (8.26) 
and the definition of Active(0). For the inductive step, 


> wi41(G, Po) 


(q1,;P2)€ Active (i+1) 


~ » » wig, Pz) Wg ps(M, P2) 
(91,P2)€Active(i+1) (q),P3)E Active (i) 

= » wi(G, P2) 

(9) ,P4)€Active(2) 

lL, 


where the first step follows from Equation (8.29), the second step follows from the fact that 
wy py isa weight function that is non zero only in pairs of Active(2 +1), and the third step 
follows from induction. Let 


o r 
wlan, P2) PH, [tri] Palga|P yp 2"? . (8.30) 


4 


WasPs try a2(%) q 


Consider a state gg of active. Then the transition enabled from q is 


~ » (8.31) 


(94 ,P5)€ Active try eQht 44 EQS, |g2 € active (q) ,P5,t71,94) 
ty 


Po (q2) Lacts( Ma) War pt tr qi (42) /W (42) (Ou Prirsat(@) t acts(Mz)) ; 


a P2tr1 4) 
where W(s2) = 1, and for each gq F# 89, 


Wm) 2 » » Wo Pliny qh (42)+ (8.32) 


(9) ,P4)€ Active tr ce" gh EQS |g #92 92 E reach (q},P5,tr1 94) 
q¢ 
1 


It is easy to verify that Expression (8.31) denotes a valid transition of a probabilistic execution 
fragment of M since it is the combination of legal transitions of a probabilistic execution 
fragment of M. The fact that the projection of a legal transition of a probabilistic execution 
fragment of M onto acts(M) is still a legal transition of a probabilistic execution fragment of 
M follows from the fact that M is a simple probabilistic automaton. 
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Informally, the set active is used to identify all the states of Hj. The transition enabled from 
each one of those states, say go, is due to several states of H,, and each state of H, influences 
the transition enabled from a specific state of Hj with a different probability. Such a probability 
depends on how much a state of H2 represents a state of H,, on the probability of the transition 
of M, that has to be matched, on the probability of reaching a specific state g5 of H2 during 
the matching operation, on the probability of reaching q2 from g$, and on the probability of 
departing from q.. These conditions are captured by PO py tryap (a2) ACO Ma Wot 75 trgh (42). 
These weights must be normalized with respect to the probability of reaching q2, which is 
expressed by W(q2). The condition gi # q in the third sum of (8.32) is justified by the fact 
W (q2) is the probability of reaching qo. 

This completes the definition of Hz, m(i), S(i), and the w,’s. We need to show that 
(H,, H2,w,S) is an execution correspondence structure via R. Thus, we need to show the 
following properties. 


1. For each i, m(2) is a fringe of Ho; 
2. For each t, m(i) < m(z4+ 1); 
3. For each state ¢ of Ho, limjsoo Vyeng<g Pld] = PalCal; 


4. For each 7, m(t) = Vopesci) Psi [PIP: 


5. For each i, fringe( 1,7) Cr S(t) via w;. 


6. For each t, each q € fringe(H,,7), and each q € states(H2), if Wi(q,q) = 0 for each 
prefix or extension qj of q2, then, for each extension qj of q such that gq) € fringe(H,,i+1) 
and each prefix or extension 9 of q2, Wi4i(qy,q4) = 0. 


We show each item separately. 


1. For each i, m(2) is a fringe of Ho. 


By construction m(z) is a probability distribution. Thus, we need to show only that for 
each state go of Ao, 


Pm ll $ Pr[Coe] (8.33) 


95 EQm( 8) 142 S95 
First we show that for each q2 € states( H2), 

W(q2) = Pr [Ca]; (8.34) 
then we show that for each q2 € states( Hz), 


Yo Priya) < W(a2)- (8.35) 


95 EQmi i) [G2 S95 


The proof of (8.34) is by induction on the length of g. If gg = s2, then (8.34) holds by 
definition. Otherwise, let gz be go without its last action and state, i.e., go = qoas for 
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some action a and some state s. Then, from the definition of the probability of a cone, 
induction, Equation (8.31) and an algebraic simplification, 


Pr, [Cq] = » » s 


(q4,P5)€ Active i GF 94 €Q5 |G2€ active(q) ,P3,tr1,95) 


1€Q 5 
Wop rsirsgh (G2) PO yp, 40, of (2) [2 (8.36) 


OW pltr, of 
From Equation (8.30) and the definition of Pp, “2 (cf. Section 4.2.7), we obtain 


PalCel= de s 
(94 ;P5)€ Active trent 44 EQS, [G2 E active(q) ,P35,tr1,95) 


f f f 
Gq P53 th1% 


! oO 
w(q),P3) Pr, [tri] Pago] Pas (8.37) 


Observe that gf € 94 and @ € active(g,,P5,tri,g) iff @ € %, & FA @, and @ € 
reach(qi, P3, tri, q). Thus, from Equation (8.31), 


Pr, [Cy] = S- S- S- Wat pling, (@): (8.38) 


(41 ,P3)€ Active rent, 9 £25 |93 #9242 E reach (44 ,P3,t71 99) 
At this point Equation (8.32) is sufficient to conclude the validity of Equation (8.34). 


The proof of Equation (8.35) is also by induction. If i = 0, then the result follows directly 
from the fact that a fringe is a probability distribution. Otherwise, let N(q1) be true iff 
q does not end in 6. Then, from Equation (8.27), 


> Pr(itt)l@] (8.39) 


95 EQm(é-41) 192 $45 


can be rewritten into 


» » wild, P2) Po P2[a)- (8.40) 


95 EQm(i-41) 192 S45 (11,P2)EActive(t) 


From the definition of P,,,p, (Equations (8.21) and (8.25)) and the definition of P,,p,1,, 
(Equation (8.17)), Expression (8.40) can be rewritten into 


S- S- Ss SS (8.41) 
93 Mm i 41) 192595 (91, P2)€Active(t),.N(q1) try EQ, 1 EN2 
wig, P2) PH, [tri] Pola) Pay Po tr qi [4] 


+ > S- w7(q1, P2) Pi, [6] Pola] 


GFE Qmeita a2 <5 (41,P2)€ Active (i),N (a1) 


+ S~ ds wil 16, Po) Poland. 


9 8EQ m(i41) 192 <95 (G18,P2)€ Active (i) 
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By exchanging sums in Expression (8.41), we obtain 


» ~ » » (8.42) 


(91;P2)€Active(z),N(a1) try Ene, a EQ2 45 EQ mci4r) l92 Sah 
. q1 Wt i 
wig, P2) Pr, [tri] Pala] Poy Petri ql] 


4 S- S- wig, Pe) PH, [6] Po[q] 


(41,P2)EActive(t),N(4q1) 95 6€EQm(i41) 192 <95 


+ - w;(q16, P2) Palq36], 


(416,P2)€Active(t) 6E€Qmci41) le2<95 


where the first summand comes from the first summand of (8.22), the second summand 
comes from the second summand of (8.22), and the third summand comes from (8.25). 
Consider the first summand of Expression (8.42), and partition the states g' of Q2 into 
those that include gq (q2 < q) and those that do not. In the first case, since from (8.27), 


(8.21), and (8.17), Qe Prtrigy © Qm(it1), and since each element qi, of Qa, Potrigf Satisfies 
qa S qa, 
» Pa Potri gil] =1; (8.43) 
95 EQm(é-41) 192 $45 
. . Oo, Py try gi! . 
in the second case the same sum gives P,, °. Consider the second summand of 


Expression (8.42), and observe that, from (8.27), (8.21), and the definition of ép,, q56 € 
mitt) 2 SG, and Po[q] > 0 iff gq € Qe, ~ < g@, and P2[q] > 0. Finally, consider 
the third summand of Expression (8.42), and observe that all the states of Q2 end with 6, 
and, from (8.27) and (8.21), @é € Qmeiti), G2 < @, and Py[q56] > 0 iff qo € Qe, @ < GS, 
P2{q56] > 0. By combining the observations above, Expression (8.42) can be rewritten 
into 


» Yo wa, Po) PH, [tr] (8.44) 


(q1,P2)EActive(t),N(a1) try EOF, 


oO ry aft 
( ~~ Plal+ nae 


gy EQ2|92 <8 ay €Q2 93 <42 


+ » Y> wig, Po) PH [6] Pola] 


(q1;P2)€Active(t),N (a1) f/EQ2|¢2<ay 


4 S- S- wilgd, P2)P2[q3]. 


(q16,P2)€ Active(2) gf EQ2 |qo<ae! 


By regrouping expressions and simplifying, we obtain 


O ra glt 
> YS wig Pa) PH Cri) Pola Pp?" (8.45) 


(91,P2)€ Active(2),N(q1) try EQ gy EQ2|92 <a4! 


+ Yo wil, P2) Palas]. 


(q1,P2)€ Active(t) qf E22 |g2<ay 
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Finally, from Equation (8.30), Expression (8.45) can be rewritten into 


» Ss > Wa petra (42) (8.46) 


(91,P2)€ Active(2),N(q1) try EQ gy EQ2|92 <a4! 


+ Yo wil, P2) Pal]. 


(q1;P2)€Active(2) 94 EQ2|q2<ay 


We now analyze the second summand of Expression (8.46), and we show by induction on 
i that it is Oif i= 0 and q F 52, it is Lift =0 and @ = 82, and it is 


» » » > Wa Potriay (4G) (8.47) 


I<t (q1,P2)€Active(7) ir EQ qh EN2 |g <2 


otherwise. For i = 0 the result is trivial. Otherwise, from Equation (8.29), 


» dD wig Gn, P2) Pola] (8.48) 


(91;P2)€Active(t+1) gf EQ2|g2<a4! 


can be rewritten into 


» » do wilds P3) wg pe (q, P2) Palas (8.49) 


(q1;P2)€Active(i+1) (q},P5)eActive(2) gf EQ2|qo<a4 


From the definition of wap, (Equations (8.23) and (8.25)), Expression (8.49) can be 
rewritten into 


S- S- S- S- (8.50) 


(q1,P2)€Active(i+1) (9},P5)e€ Active(t),N(q}) 4 94 €Q2|92<94 
Ay 


wild), Po) Pat, [tr war pr ert (11 Po)) Palag] 


> » 


(q16,P2)€Active(i+1) (9) ,P5)€Active(i),N(q)) g3 EQ2|¢2 <as! 
wild, Ps) PH, [4] wsqi p1 (416, Po) Pal gy] 
+ wil a6, Ph) Pola. 


(91 5,P5)€Active(t) qf EQS |ao<ay 


tr EQ 


Observe that in the first summand of (8.50) 
» do Wg hers (1, P2) Pola] 
(q1;P2)€Active(t+1) gf EQ2|q2<a4 


= S " 
= > Po print [P2]P2[q3] 
P2|591,(¢1,P2)€ Active (it+1) gf €Q2|q2<94! 


= S- S- Pups tri ay’ [az], 


a / Me Me 
qs EQS qF EQ rt aorept |go<@ 
2 2 42 q Ps try | 12 
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where the first step follows from the fact that Wa! Pl tr! git! is a weight function, and the 
second step follows from (8.17), (8.15) and the fact that Q,p:;,1 is the set of probability 
space Pz such that there is a state q, where (q,P2) € Active(t + 1) (cf. the definition 
of Active and observe that |q;| = 2+ 1). For the second summand of (8.50), observe 
that for each pair (q6, P2) of Active(i + 1), if Pi. [6] > 0, then there is exactly one pair 
(M1, P3) of Active(?) such that wep: (q,6, P2) > 0. In particular, q = gj, P2 = dp, and 
Wsqtp1(G16, Pz) = 1. Conversely, for each pair (q),P}3) of Active(i) such that Pit [6] > 0, 
the pair (q6, P2) is in Active(i+1) and wsy p:(q16, P2) = 1. Thus, the term ws, p:(q4, P2) 
and the sum }i(5,pP))eActive(it1) Cat be removed from the second summand of (8.50). 


Thus, by applying the observations above to (8.50), we obtain 


» yy ~ (8.51) 


(af PS EActIVE)N() 1 eg AEM, BEM pr yt gr laeS af 


/ 
try EQy, 


wild), Po) Pat, [tr Pala Pa pant aul a2] 


+ S- So wilai. PS) Py [P3106] 


(44 ,P5)€Active(?),N(q1) 94" EQS |a2<ay 


+ > S> wid, PS) P5[a. 


(94. 6,P5)€ Active(2) gf EQS4 |qo<qh" 


Consider the first summand of Expression (8.51). If qo < qf’, then 


» Porphert qld] = 1; (8.52) 


Me Me 
QZ EQ st arapt 1 |g2<@ 
2 Gq, P35 th % | 72 


If gf! < qo, then 


Ao 


a (8.53) 


Oras 
mM GP tr. 
5 f of Phir! i'l | f q2 


9 EQY pl ert git V2 99 
Thus, from Equations (8.52) and (8.53), Expression (8.51) can be rewritten into 


> S> wilgh, PL) PS [trl] (8.54) 


(9) ,P5)€Active(i),N(q)) trl ent 


Oy pltrt git! 
( mus YS naar) 
g 


BES |g2<ay" a3 €Q8 lay" <2 


+ S- Yo wil. PS) PH [P31] 


(a, ,P$)€Active(i),N(44) 9f”€EQ5 |a2<a4 


+ S- S- wil ad, PS) P54". 


(94. 6,P5)€ Active(2) gf EQS4 |qo<qh" 
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By regrouping the subexpressions in (8.54), we obtain 


f oO; ttyl at 
» SE wil PSP (ert Pale)" (8.55) 


(91 ,P3)€Active(?),N (91) a 9y"EN5|q3"<92 


Ay 


i i Apu 
+ yy yy wi(q, Po) Polqo']- 
(94 ,P5)€ Active(2) gf EQS4 |qo <4" 


tr EQ 


From Equation (8.30), Expression (8.55) can be rewritten into 


S- S- Yo Wor part qttt(@2) (8.56) 


(9) ,P5)€Active(i),N(q)) at gE, [al <an 
Fy 


+ S- S> wil. P3) P3461. 


(9) ,P4)€ Active(2) gf EQS |q2<qh" 


tr, EQ 


The induction hypothesis is now sufficient to conclude the validity of (8.47). From an 
alternative characterization of the set {qf € Qe | q¥ < qo} in Expressions (8.47) and (8.45), 
and by combining (8.45) and (8.47), we obtain 


> Pr(itl@] (8.57) 


95 EQm(é-41) 192 $45 


= » » » » Wo Potrigit (2): 


IS (q1,P2)€Active(3) trp QT, 9 EN2| 99 #2 ,4 E reach (91 P2, 11,92) 


Observe that the right expression of (8.57) contains a subset of the terms of the right 
expression of Equation (8.32). This is enough to conclude the validity of (8.35). 


. For each t, m(t) < m(i+ 1). 


This result follows directly from Equation (8.57). In fact, for each state q2 of Hz, Ex- 
pression (8.57) for m(z + 1) contains a subset of the terms of the Expression (8.57) for 


. For each state ¢ of Ho, limjioo Vyeajiq<q Pil’) = PalCy- 


This result follows directly from Expression (8.57). In fact, as 7 — oo, the right expression 
of (8.57) converges to the right expression of (8.32). 


. For each i, m(2) = DP esti) Psiy[PIP. 


For 7 = 0 the result is trivial. For i > 0, from Equation (8.27), m(i+ 1) is rewritten into. 


» wig, P2)PaPo- (8.58) 
(q1,P2)€ Active (2) 


From Equation (8.21), Expression (8.58) can be rewritten into 


dwn Pa) {Pi ftrs|Parpaire + Piz l6l6r2 | - (8.59) 
(q1,P2)€ Active (2) rE} 
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From Equation (8.17) applied to P,,p,1-, and Equations (8.15) and (8.19) applied to 
Pi, [élép,, Expression (8.59) can be rewritten into 


YS wil, Pa} SD Plt | do Pop, [PIP | + (8.60) 


wel; “ 3 
(q1,P2)€ Active(t) tr EQyy, PED wo try 


Pit [6] 3) Psp, [PIP 
P EN5,, 


From Equation (8.22), Expression (8.60) can be rewritten into 


S> wig, P2)} So Pep, [PIP]. (8.61) 
(q1,P2)€ Active (2) PEQS p, 


Finally, from Equation (8.28), Expression (8.61) can be rewritten into 


S> Psigty[PIP, (8.62) 
PEQ S41) 


which is what we needed to show. 


. For each i, fringe( Hy, 71) Cr S(t) via vj. 
For 2 = 0 the result is trivial. By applying the definitions of a fringe and of fringe(H,i+1), 
fringe( 41,1 + 1) 
= S- Pr, [Co lPa, 
qi €states( H2)||¢2|=tor g2=956,|¢2|<? 


= S- wil, P2)Pq - 
(q1,;P2)€ Active (2) 


From (8.28), 


S@+1)= S- wi(q, P2)Pe p,- 
(q1,;P2)€ Active (2) 


Since for each pair (q1,P2) of Active(i), Py, Er PP p, via wa,,p,, from Lemma 8.2.1, 


> wilqi, P2)Po, Er > wi(qi, Pa)Prp, 
(q1,P2)€ Active(i) (q1,P2)€Active(é) 


Via Y2(q1,P2)eActive(i) Wil G1, P2)Wg,P., Which is wi41. = 
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6. For each t, each q € fringe(H,,7), and each q € states(H2), if Wi(q,q) = 0 for each 
prefix or extension qj of q2, then, for each extension qj of q such that gq) € fringe(H,,i+1) 
and each prefix or extension 9 of q2, Wi4i(qy,q4) = 0. 


Suppose by contradiction that there is an extension qj of q such that qj € fringe(H1,i+1) 
and a prefix or extension 4 of gz such that Wi41(q),95) > 0. From the definition of W; 
and Equation (8.29), 


Wisin, %) = >> > wilh, P2)wa,P2(n, P)P[d)- (8.63) 
P (%1,P2)€Active(2) 


Since W;(q1, q5) > 0, then there is at least one probability space P and one pair (G1, P2) € 
Active(i) such that w;(q@i,P2) > 0, waq,p.(m,P) > 0, and Pl[qi] > 0. Then there is at 
least one prefix gf of qi such that P2[q] > 0, which means that W;(qj1, qf) > 0. However, 
this is a contradiction since gf is either a prefix or a suffix of qo. 


The execution correspondence theorem can be stated and proved similarly for weak and strong 
probabilistic simulations. The proofs are simpler than the proof presented in this section, and 
thus we omit them from this thesis. 


8.6.4 Transitivity of Probabilistic Forward Simulations 


Now we have enough machinery to prove that probabilistic forward simulations are transitive, 
e., if My, Ers Mz and Mz Ers Ms, then My Ers M3. We start by proving a lemma. 


Lemma 8.6.2 Let (Hy, H2,m,5') be an execution correspondence structure via the probabilistic 
forward simulation R, and suppose that H, represents a weak combined transition s —>c¢ Py. 
Then Hz represents a weak combined transition s' + ¢ Pz and there is a probability space P? 
such that 


1. Pi Cr PP and 
2. Po = Dregs Pz [PIP. 


Proof. Let w; be the weight functions for fringe(H1,1) Cr S(2). Let Pl be 6-strip(PxH,), P5 
be 6-strip(Px, ), and let 


Pos = 2 Whajer(d, PP. (8.64) 


adeQ py, Plwlal4r (a6,P)>0 


For each aé € Qy, and each P € Probs(extstates(H2)), let w(ad,P) = = wiajer(ad, P). 

We show that w is a weight function from P; to P} ¢ and that P} ¢ is well defined. This im- 
plies that P| Er P35. Then we show that for each element aé of Qu, pen, . P3 5[P]Plaé] = 
Py,[Cos]. Since all the elements of the vane spaces of 5 < end with é, we obtain that 
P3, is well defined and that P5 = PEM, « > s[P|P. Then the lemma is proved by defining Py 
to be Istate(P{), Pz to be Istate(P3), and a to be Istate(P5 5). 

To show that w is a weight function we have to verify the three conditions of the definition 
of a weight function. If w(aé,P) > 0, then, from the definition of w, wyaj4i(aé,P) > 0. 
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Since waj41 is a weight function, then ad R P. Let P € 5 s- Then yaseOn, w(aé,P) = 
aden, Wio|41(06, P), which is P; s[P] by definition of Pj >. Consider now an element aé of 
Qy,. Then, UP EM, « w(a6, P) = UP en « Wo|41(a6, P). Since w),)41 is a weight function, then 
the sum above gives Py, [Cas] = Pj[aé]. To show that P3 ; is well defined we need to show that 
yoseNn, DP lwjatyr (08,P)>0 Wo|41(a6, P) = 1. This follows immediately from the fact that w is a 
weight function and that, since H, represents a weak combined transition, yoseNn, Pi[a6] = 1. 

We are left to show that for each element a6 of Qx,, S’peas . Py s[P|P[aé] = Pr,[Cosl.- 


Observe that for each element a6 of Qy,, if i < Jal then w;(ad, P) is undefined for each P, and 
if > Jal, then for each 7 > i and each P, w;(a6,P) is defined iff w;(aé,P) is defined, and if 
w;(ad, P) is defined then w;(aé,P) = w;(a6,P). Thus, if we extend each w; by setting it to 0 
whenever it is not defined, then, for each aéd € Qy,, 


S> Pi s[P]Pled]= S- (in S- wos?) Plaé]. (8.65) 


i i Too &§ Q 
PEN, PEM, « abEQH, 


Since for each 2, w; is a weight function, and since from the definition of P} , each element P 
for which w;(a6,P) > 0 is in 4 ¢, then we derive 


S” Pis[P]Plaé)= S- (Jim Ps«lP) Plaé). (8.66) 
PED ¢ PED ¢ 


By exchanging the limit with the sum and by using Condition 3 of the definition of an execution 
correspondence structure, the equation above can be rewritten into 


S> P3,s[P]Plad] = jim m(i)[ad), (8.67) 
PED ¢ 


which gives the desired result after using Condition 2 of the definition of an execution corre- 
spondence structure. a 


Proposition 8.6.3 Probabilistic forward simulations are transitive. 


Proof. Let Ry, be a probabilistic forward simulation from My, to Mo, and let R2 be a proba- 
bilistic forward simulation from Mp to M3. Define R so that s, R P3 iff there is a probability 
space Pz, and a probability space P}, such that 


1. S| Ry P2, 


2. P2 CR, PS, and 


We need to show that R is a probabilistic forward simulation from My, to Mz. For this purpose, 
let sy R Pz, and let P2 and PP satisfy the three conditions above. Let s; —+ P,. Let M3 
be obtained from My by introducing a new state s, and by adding a transition s, ++ Py, 
where 7 is an internal action; similarly, let M4 be obtained from M3 by introducing a new state 
s, and by adding a transition s4 + P3, where 7 is an internal action. Let R{ be obtained 


190 


from Ry by adding the pair (s1,DP(s%)), and let R5 be obtained from Rz by adding the pair 
(sh, D(s5)). Observe that #4 is a probabilistic forward simulation from My, to Mé and that R4 
is a probabilistic forward simulation from M3 to M3. 

We want to find two probability spaces P4 and P3 ¢ such that s4 +c Ph, Pi Er P35) 
and P5 = Yipes , P3,s[P]P. From the definition of a weak transition, this is sufficient to show 


that for each state s of P3 there is a weak combined transition s $c P, of Mz such that 
Ps, = yi sENs P3[s|Ps. 

Since R} is a probabilistic forward simulation, there is a weak combined transition s4 —>¢ 
P3, of M3 and a probability space P4 , such that 


PL= Dd) Pis[P]P and Pi Cr, Pi.s. (8.68) 
PED 5 


Let Hz be the probabilistic execution fragment of Mj that represents the weak combined tran- 
sition s —+c P}. Then, by definition of Hz, Pi = Istate(6-strip(Py,)) (cf. Section 4.2.7). 

From the Execution Correspondence Theorem there is an execution correspondence struc- 
ture (H2, H3,m, 5), where Hz is a probabilistic execution fragment of M3 that starts from s%. 
From Lemma 8.6.2, H3 represents a weak combined transition s4 —>+c P4 for same probability 
space P4. Moreover, there is a probability space Ps such that 


PJ= DY) PJs[P]P and Pi Er, PSs. (8.69) 
PEQ 5 


Let w2 be the weight function for P, Cr, Psis. For each probability space P of 05 g, let 
wp : states( M2) x Probs(states(M3)) — [0,1] be a function that is non-zero only in the set 
Q x O35 and such that for each pair (s,P’) of Q x 03 5, 


P[s]wo(s, PY) 


mele P) = ~~ pag 


(8.70) 


Also, for each probability space P of Qs let 


PSs = S> So wp(s,P')D(P’), (8.71) 


s€Q PEN" 
and let 
PSS So PPS IP’. (8.72) 
PIE? g 


Let Ps 5 be the discrete probability space where 03 ¢ = {PP | P € Qos}, and for each element 
P P) ; ; 
Ps of 25 5, Ps s[P3] = prea, .|PP=PP" P35[P’]. Then, the following properties are true. 


1. For each probability space P of 5 $5 wp is a weight function from P to P35. 


We verify separately each one of the conditions that a weight function must satisfy. 
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(a) For each s € states( Mp), P[s| = > Pe Probs(states(M3)) wp(s, P’). 
From the definition of wp, the right expression above can be rewritten into 
Pls|wo(s, P) 


Pid (8.73) 


P'EProbs(states(Ms3)) 


Since wz is a weight function, pre prots(states(Ma)) W2(S,P’) = P3[s], and thus Ex- 
pression 8.73 becomes P[s]. 


(b) For each P’ € Probs(states(Ms)), Dosestates(Ma) WP(S, P’) = PP<[P'- 
From Equation (8.71), P?s[P’] = seq wr(s,P’). Since wp is non-zero only when 
the first argument is in Q, PP [P| = Vsestates(My) WP(S, P’). 
(c) For each (s,P’) € states(M 2) x Probs(states(M3)), if wp(s,P’) > 0 then s Ry P’. 
If wp(s,P’) > 0, then, from Equation (8.70), w2(s,P’) > 0. Since we is a weight 
function, then s R2 P’. 
2. ren, , P3,s[P|P = Pe. 


From the definition of P35, Equation (8.72), Equation (8.71), and Equation (8.70), 
pen, P3,5[P]P can be rewritten into 


~ » Pr slPl——prgy — P (8.74) 


PED ¢ PIEQ 5 s€states(M>) 


From (8.68), Expression (8.74) can be rewritten into 


SY Beles PY py (8.75) 


Pils 
P'EQL . s€states(Me) 2 


After simplifying P[s], since w2 is a weight function from Pj to Py, Expression (8.75) 
can be rewritten into 


dS PSslP TP’, (8.76) 
PIEQ! ¢ 


which can be rewritten into PY using Equation (8.69). 
3. For each pair (s,,P) such that si; Ri P, si, Rs PP. 
This follows directly from 1 and (8.72). 


Let PS be PY, and define a new weight function w : states(M,) x Probs(states(M3)) — [0,1] 
such that, for each probability space P of 04 55 w(s1, PZ) = wi(s1,P). Then, it is easy to check 
that P; Cr P35 via w. This fact, together with 2, is sufficient to complete the proof. | 
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8.7 Probabilistic Forward Simulations and Trace Distributions 


In this section we show that probabilistic forward simulations are sound for the trace distribution 
precongruence. Specifically, we show that MJ, Erg Mz implies M, Ep Mz. Thus, since Epg is 
a precongruence that is contained in Ep, from the definition of Epc we obtain that M, Ers Mo 
implies My Epc M2. 


Proposition 8.7.1 Let M, Crs Mz. Then M, Cp Mo. 


Proof. Let R be a probabilistic forward simulation from My, to M2, and let Hy be a proba- 
bilistic execution of M4, that leads to a trace distribution D,. From Lemma 8.6.1, there exists 
a probabilistic execution Hz of Mz and two mappings m,.$ such that (Hy, H2,m,5) is an exe- 
cution correspondence structure for R. We show that H2 leads to a trace distribution D2 that 
is equivalent to D,. 

Consider a cone C’g of D,. The measure of Cg is given by 


Pr, [Cq,]- (8.77) 
q1 € states( Hy )|trace(q1)=G,lact(q1)=lact(G) 


The same value can be expressed as 


lim S- Pr, [Ca]. (8.78) 


qi€fringe( Hy ,t)|C<trace(q1) 


Consider a cone C’g of Dz. The measure of Cg is given by 


PH, [Co]. (8.79) 
q2 € states(H2)|trace(q2)=G,lact(q2)=lact(G) 
The same value can be expressed as 
lim S- Pr(ilCoa]- (8.80) 


I-00 


q2€Em(t)|B<trace(q2) 


The reason for the alternative expression is that at the limit each cone of Expression (8.79) is 
captured completely. Thus, it is sufficient to show that for each finite @ and each 7, 


» Pu, [Ca] = S- Pri) . (8.81) 


qi €fringe(Hy ,2)|G<trace(q1) q2€Em(t)|B<trace(q2) 


This is shown as follows. Let w; be the weight function for m(i) Er $(i). Then, 


S- Pr [Cy] = S- SY) wilt, P2). (8.82) 


q€fringe( Hy ,t)|8<trace(q) qi €fringe (Hy ,i)|G<trace(q1) P2€S(%) 


Observe that each probability space of $(2) has objects with the same trace, that each state q¢ 
of fringe(H,, 7) is related to some space of $(z), and that each space of $(7) is related to some 
state q of fringe( H,,7). Thus, from (8.82), 


> Puy, [C,] = » Dd, wilt, P2)- (8-83) 


q€fringe(Hy ,2)|G<trace(q) P2ES (i) |Aqs eng F< trace(g2) a1 €fringe (Ai, i) 
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Since w; is a weight function, we obtain 


Py, [Cq] = > Psy[Pal. (8.84) 
q€fringe(Hy ,2)|G<trace(q) P2€S (i) |Aqo ens BS trace (qa) 


Since in a probability space the probability of the whole sample space is 1, we obtain 


> Pr, [Cq] = yy 2 Psciy[P2] Pola]. (8.85) 


q€fringe(Hy ,2)|G<trace(q) P2€S (i) |Aqg ens BS trace(g2) 2EN2 


From an algebraic manipulation based on Condition 3 of an Execution Correspondence Struc- 
ture, we obtain 


Py, [Cq] = > S> Psi [Pa] Palae)- (8.86) 
q€fringe(Hy ,2)|G<trace(q) q2€Em(t)|B<trace(q2) P2€S(t)|q2EQe 


Finally, from Condition 3 of an Execution Correspondence Structure again, we obtain Equa- 
tion (8.81). = 


8.8 Discussion 


Strong bisimulation was first defined by Larsen and Skou [LS89, LS91] for reactive processes. 
Successively it was adapted to the alternating model by Hansson [Han94]. In this thesis we 
have defined the same strong bisimulation as in [Han94]. The formal definition differs from the 
definition given by Hansson in that we have used the lifting of a relation to probability spaces 
as defined by Jonsson and Larsen [JL91]. 

Strong simulation is similar in style to the satisfaction relation for the probabilistic specifi- 
cation systems of Jonsson and Larsen [JL91]. It is from [JL91] that we have borrowed the idea 
of the lifting of a relation to a probability space. 

The probabilistic versions of our simulation relations are justified both by the fact that a 
scheduler can combine transitions probabilistically, as we have said in this thesis, and by the fact 
that several properties, namely the ones specified by the logic PCTL of Hansson and Jonsson 
[Han94], are valid relative to randomized schedulers iff they are valid relative to deterministic 
schedulers. This fact was first observed by Segala and Lynch [SL94] and can be proved easily 
using the results about deterministic and randomized schedulers that we proved in Chapter 5. 

The weak probabilistic relations were introduced first by Segala and Lynch [SL94]. No 
simulation relations abstracting from internal computation were defined before. Probabilistic 
forward simulations are novel in their definition since it is the first time that a state is related 
to a probability distribution over states. 
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Chapter 9 


Probabilistic Timed Automata 


9.1 Adding Time 


So far we have extended labeled transition systems to handle probabilistic behavior; however, 
we have not addressed any real-time issue yet. The main objective of this chapter is to add 
time to probabilistic automata. 

Following an approach that Abadi and Lamport [AL91] call the “old-fashioned recipe”, we 
address real-time issues by augmenting probabilistic automata with some structure that models 
passage of time. In particular, we adopt the solution of Lynch and Vaandrager [LV95], where 
a timed automaton is an ordinary automaton whose actions include the positive real numbers. 
The occurrence of a real number d means that time d elapses. In addition, a timed automaton 
of [LV95] is required to satisfy two trajectory axioms: the first axiom says that if time d can 
elapse and immediately afterwards time d’ can elapse, then time d+ d’ can elapse; the second 
axiom says that if time d can elapse, then there is a trajectory that allows us to associate every 
real time in the interval [0,d] with a state. 

The introduction of real-time in probabilistic automata presents two main problems. 


1. Time is a continuous entity, and the time that elapses between the occurrence of two sep- 
arate actions may depend on a probability distribution that is not discrete. For example, 
the response time of a system may be distributed exponentially. On the other hand, the 
probability distributions that we allow in the untimed model are only discrete. 


2. In the untimed model the parallel composition operator is defined only for simple prob- 
abilistic automata. Since time-passage is modeled by actions of Rt, in a simple proba- 
bilistic timed automaton it is not possible to let time pass according to some probability 
distribution. 


The first problem could be solved by removing the requirement that the probability distribution 
associated with a transition is discrete. However, in such case we would need to redevelop the 
whole theory, while if we force each probability distribution to be discrete we can reuse most 
of the results of the untimed model. For this reason, we choose to work only with discrete 
probability distributions and we defer to further work the extension of the model to non-discrete 
probability distributions (cf. Section 13.2.1). 
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For the second problem the reader may object that it originates from the choice of using 
a distinct time-passage action for each amount of time that elapses in a transition, and thus 
we may conclude that the problem would be solved by using a unique action that expresses 
passage of time [LV93b] rather than a different action for every time; however, the problem has 
deeper roots. 


Example 9.1.1 (Problems with probabilistic passage of time) Suppose that from state 
8, a probabilistic timed automaton M, lets time pass for 1 second with probability 1/2 and for 
2 seconds with probability 1/2 before performing an action a, and suppose that from state s2 a 
probabilistic timed automaton Mg lets time pass for 0.5 seconds with probability 1/2 and for 1.5 
seconds with probability 1/2 before performing action a. What is the probability distribution 
on the time that elapses from state (s1, 82) of M,||M2 before performing a? What can we 
say about the projections of a probabilistic execution of M;||M2? The reader may note the 
similarity with the problems encountered in the definition of parallel composition for general 
probabilistic automata (cf. Section 4.3.3). = 


In order to simplify the handling of trajectories, in this thesis we impose an additional restric- 
tion on the time-passage transitions of a probabilistic timed automaton; namely, each transition 
involving time-passage is required to lead to a Dirac distribution. Probabilistic behavior as- 
sociated with passage of time is allowed only within a probabilistic execution. Even though 
this timed model may appear to be restrictive, it is sufficiently powerful to analyze non-trivial 
timed properties of randomized algorithms (cf. Chapter 10). 

In the rest of this chapter we concentrate on the definition of the timed model as an extension 
of the probabilistic automata of Chapter 4. Most of the concepts are extensions of the definitions 
of [LV95] to the probabilistic framework; the non-trivial part of the chapter is the definition of 
a probabilistic timed execution, where some measure-theoretical complications arise. 


9.2 The Timed Model 


In this section we define probabilistic timed automata as an extension of the probabilistic 
automata of Chapter 4, and we extend the timed executions of [LV95] to our framework. Due 
to the complications that arise in the definition of a probabilistic timed execution, we define 
probabilistic timed executions in a separate section. 


9.2.1 Probabilistic Timed Automata 


A probabilistic semi-timed automaton M is a probabilistic automaton whose set of external 
actions includes #t, the set of positive reals, and whose transitions with some action in RT 
are non-probabilistic, i.e., they lead to a Dirac distribution. Actions from #T are referred to as 
time-passage actions, while non-time-passage actions are referred to as discrete actions. We let 
d,d',... range over Rt and more generally, t, t’,... range over the set RU {oo} of real numbers 
plus infinity. The set of visible actions is defined by vis(M) = ext(M)\ Rt. 

A probabilistic timed automaton is a probabilistic semi-timed automaton M that satisfies 
the following two axioms. 


d d’ d+d! 
Al If s — s’ and s’ —> 8", then s are sll, 
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For the second axiom, we need an auxiliary definition of a trajectory, which describes the 
state changes that can occur during time-passage. Namely, if J is any left-closed interval of 
beginning with 0, then an J-trajectory is a function w: J — states(M), such that 

w(t) = w(t’) for all t,t! € I with t < t’. 
Thus, a trajectory assigns a state to each time ¢ in the interval J in a “consistent” manner. We 
define Itime(w), the “last time” of w, to be the supremum of J. We define fstate(w) to be w(0), 
and if J is right-closed, we also define Istate(w) to be w(ltime(w)). A trajectory for a transition 


s+ slisa [0, d]-trajectory such that fstate(w) = s and Istate(w) = s’. Now we can state the 
second axiom. 


A2 Each time-passage transition s 1. st hasa trajectory. 


A probabilistic timed automaton M is simple if M is a simple probabilistic automaton. 

Axioms Al and A2 express natural properties of time: Axiom A1 says that if time can 
elapse in two transitions, then it can also elapse in a single transition; Axiom A2 says that if 
time d can elapse, then it is possible to associate states with all times in the interval [0,d] in a 
consistent way. 


Example 9.2.1 (The patient construction) A simple way to add time to a probabilistic 
automaton is to add arbitrary self-loop timed transitions to each state of a probabilistic au- 
tomaton. Specifically, given a probabilistic automaton M, we define patient(M) to be the 
probabilistic timed automaton M’ such that 


1. states( M’) = states(M), 

2. start(M’) = start(M), 

3. acts(M') = acts(M)URT, 

4. trans(M') = trans(M)U {(s, d,s) | s € states(M),d © RT}. 


Thus, patient(M) is like M except that an arbitrary amount of time can elapse between two 
discrete transitions. It is immediate to verify that patient(.M) satisfies axioms Al and A2. 
The patient construction was first defined for ordinary automata in [VL92]. a 


Example 9.2.2 (Simple restrictions on time passage) The patient construction does not 
specify any limitations to the way time can elapse. Sometimes we may want to specify upper 
and lower bounds to the time it takes for some transition to take place. Such a limitation can 
be imposed easily by augmenting the states of a probabilistic automaton with variables that 
express the time limitations that are imposed. As an easy example consider a probabilistic 
automaton M with a unique state s and a unique discrete transition (s,a,s). Suppose that we 
want to add time to M and impose that action a occurs once every at least 1 time unit and at 
most 2 time units. Then the corresponding probabilistic timed automaton M’ can be specified 
as follows. 


1. states(M’) = {(s,l,h)|0<1<1,0<1<h< 2}, 
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2. start(M’) = {(s,0,2)}, 
3. acts(M') = {a} UR, 


4. trans(M’) = {((s,0,h),a,(s,1,2))|O0<h < 2}U {((s,Uh),d,(s,l-—d,h-d))|d<l< 
h}U {((s,0,h), d,(s,0,h — d))d < h}. 


The variables / and h keep track of the time that must or can elapse before performing a. Time 
passage decreases both the variables unless they are 0. Action @ can occur only when | = 0 
and leads to a state where / = 1. This means that at least 1 time unit must elapse before a 
can be performed again. No time can elapse if h = 0. At thet point the only transition that 
can be performed is the transition labeled with a. Thus, no more than 2 time units can elapse 
between the occurrence of two actions a. It is immediate to verify that M’ satisfies axioms Al 
and A2. | 


9.2.2 Timed Executions 


Since a probabilistic timed automaton is also a probabilistic automaton, the executions of the 
untimed model carry over to the timed case. However, an execution associates states with just 
a countable number of points in time, whereas the trajectory axiom A2 allows us to associate 
states with all real times. Also, our intuition about the executions of a timed system is that 
visible actions occur at points in time, and that time passes “continuously” between these 
points. In other words, at each point in time a system is in some state. This leads to the 
definition of a timed execution. 


Timed Executions 


A timed execution fragment a of a probabilistic timed automaton M is a finite or infinite 
alternating sequence, @ = WoG1W 1 dQW2---, where 


1. Each w; is a trajectory and each a; is a discrete action. 
2. If a is a finite sequence then it ends with a trajectory. 


3. If w; is not the last trajectory in a then its domain is a right-closed interval, and there 
exists a transition (Istate(w;),P) of M such that (a, fstate(wj41)) € 2. 


A timed execution fragment describes all the discrete changes that occur, plus the evolution 
of the state during time-passage transitions. If a is a timed execution fragment, then we 
let Itime(a) denote >°; Itime(w;). Note that we allow the case where the domain of the final 
trajectory is of the form [0,00); in this case Itime(a) = oo. We define the initial state of a, 
fstate(a), to be fstate(wo) 

A timed execution is a timed execution fragment whose first state is a start state. 

The timed executions and timed execution fragments of a probabilistic timed automaton 
can be partitioned into finite, admissible, and Zeno timed executions and timed execution 
fragments. A timed execution (fragment) a is finite, if it is a finite sequence and the domain of 
its final trajectory is right-closed; a timed execution (fragment) a is admissible if Itime(a) = 00; 
a timed execution (fragment) a is Zeno if it is neither finite nor admissible. 
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There are basically two types of Zeno timed executions: those containing infinitely many 
discrete actions in finite time, and those containing finitely many discrete actions and for which 
the time interval associated with the last trajectory is right-open. Thus, Zeno timed executions 
represent executions of a probabilistic timed automaton where an infinite amount of activity 
occurs in a bounded period of time. (For the second type of Zeno timed executions, the infinitely 
many time-passage transitions needed to span the right-open interval should be thought of the 
“infinite amount of activity”.) 

We will be interested mostly in the admissible timed executions of a probabilistic timed 
automaton since they correspond to our intuition that time is a force beyond our control that 
happens to approach infinity. However, according to our definition of a probabilistic timed 
automaton, it is possible to specify probabilistic timed automata in which from some states 
no admissible timed execution fragments are possible. This can be because only Zeno timed 
execution fragments are possible from that state, or because time cannot advance at all (in which 
case a time deadlock has occurred). Although Zeno timed executions are usually non-desirable, 
research experience has shown that the analysis of a model would be more complicated if Zeno 
timed executions are ruled out. 

Denote by t-frag*(M), t-frag?(M), and t-frag(.M) the sets of finite, admissible, and all 
timed execution fragments of M. Similarly, denote by t-exec*(.M), t-exec™®(M), and t-exec(M) 
the sets of finite, admissible, and all timed executions of M. 

A timed extended execution fragment of M, denoted by a, is either a timed execution 
fragment of M or a sequence a’é where a’ is a timed execution fragment of M. Denote by 
t-execs(M) and t-execs(M) the sets of finite and all timed extended executions of M. 


Concatenations, Prefixes and Suffixes 


If w is an [-trajectory where / is right-closed, and w’ is an I’-trajectory such that Istate(w) = 
fstate(w’), then w and w’ can be concatenated. The concatenation, denoted by ww” is the least 
trajectory (the trajectory with the smallest domain) w” such that w(t) = w(t) for t € J, and 
w(t + Itime(w)) = w(t) for t € I’. It is easy to show that w” is a trajectory. 

Likewise, we may combine a countable sequence of “compatible” trajectories into one: if w; 
is an J;-trajectory, 0 < 7 < oo, where all J; are right-closed, and if Istate(w;) = fstate(wi41) for 
all 7, then the infinite concatenation wyw2--- is the least function w such that for all 7 and all 
te Tj, w(t + Vj <; time(w;)) = w;(t). It is easy to show that w is a trajectory. 

A finite timed execution fragment a = woajw1-+-d,W, of M and a timed (extended) execu- 
tion fragment a! = wl dn4iwn41 ++: of M can be concatenated if Istate(a) = fstate(a’). In this 
case the concatenation, written a~ a’, is defined to be a” = woayw-- An (WW) Ong Wrpi ict 
It is easy to see that a is a timed (extended) execution fragment of M. 

The notion of prefix for timed execution fragments and timed extended execution fragments 
is defined as follows. A timed (extended) execution fragment a of M is a prefix of a timed 
(extended) execution fragment a’ of M, written a < a’, if either a = a’ or a is finite and there 
exists a timed (extended) execution fragment a” of M such that a’ =a~ a”. Likewise, a is a 
suffix of a’ if there exists a finite timed execution fragment a” such that a’ = a” ~ a. Denote 
a by a’pa”, 

The length of a timed execution fragment a expresses the number of discrete actions in 
a. Thus, even though a is admissible or Zeno (and thus not finite), its length may be finite. 
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Formally, define the length of a = woa,wyagqwe--- as 


Ja A n if ais a finite sequence and ends in w, 
al = : : : : 
oo if a is an infinite sequence. 


9.3. Probabilistic Timed Executions 


Since a probabilistic timed automaton is also a probabilistic automaton, it is possible to talk 
about the probabilistic executions of a probabilistic timed automaton. However, as we have 
pointed out already for ordinary executions, a probabilistic execution does not describe com- 
pletely the evolution of a probabilistic timed automaton since it does not allow us to associate 
every real time with the states that are reached at that time. We need a structure that extends 
probabilistic executions in the same way as a timed execution extends an execution. A timed 
execution differs from an execution in two aspects: 


1. a timed execution has trajectories to express passage of time; 
2. a timed execution does not contain any time-passage actions. 


In particular, a timed execution hides the time-passage transitions that are scheduled in an 
execution to let time pass. Given a trajectory w, there are infinitely many ways to schedule time- 
passage transitions to move in time Itime(w) from fstate(w) to Istate(w) (Istate(w) is meaningful 
only if the domain of w is right-closed); the trajectory w represents all those possible ways. In a 
similar way, a probabilistic timed execution should not contain any information on the specific 
time-passage transitions that are scheduled. Thus, a probabilistic timed execution should be 
a structure where each state records the past history and each transition contains information 
on the trajectories that are spanned till the occurrence of the next action. However, it may be 
the case that there is no next action since the next trajectory is right-open. This would not 
be a problem except for the fact that from a state there can be uncountably many right-open 
trajectories that leave even though they are generated by scheduling time-passage transitions 
according to a discrete probability distribution. 


Example 9.3.1 (Uncountable branching from countable branching) Consider a prob- 
abilistic automaton M that can increase or decrease a variable x of its state at a constant speed, 
and suppose that every one time unit the speed of x can be complemented nondeterministi- 
cally. A valid scheduler A for M is a scheduler that every one time unit chooses the sign of the 
speed of a according to a uniform binary distribution. As a result, there are uncountably many 
trajectories leaving from the start state of M if we use A to resolve the nondeterminism. Thus, 
if in a probabilistic timed execution we do not allow for a trajectory to be split into pieces, 
the probabilistic timed execution of M generated by A would have a non-discrete probability 
distribution in its transition relation. | 


To express the fact that we allow only discrete probability distributions on a scheduler, we define 
probabilistic timed executions in two steps. First we define probabilistic time-enriched execu- 
tions, which contain closed trajectories and time-passage actions (the time-passage transitions 
that are scheduled are visible); then, we remove the time-passage actions from probabilistic 
time-enriched executions to yield probabilistic timed executions. 
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At the end of this section we show that probabilistic executions, probabilistic time-enriched 
executions, and probabilistic timed executions are strongly related. Specifically, we show that 
each probabilistic execution is a sampling of a probabilistic time-enriched execution where 
the information contained in the trajectories is lost, and that each probabilistic time-enriched 
execution is sampled by some probabilistic execution. Furthermore, we show that it is possible to 
define an equivalence relation directly on probabilistic time-enriched executions that expresses 
the fact that two probabilistic time-enriched executions denote the same probabilistic timed 
execution (they just schedule time-passage transitions in a different way). 

All the equivalence results that we prove in this section allow us to use the kind of proba- 
bilistic execution that is best suited for each problem. In particular, we use probabilistic timed 
executions for the theorems of Chapter 10, and we use probabilistic time-enriched executions 
and probabilistic executions for the results of Chapters 11 and 12. Due to the purely technical 
content of the comparison section (Section 9.3.3), the reader may focus just on the definitions 
and on the informal explanations (Sections 9.3.1 and 9.3.2) at a first reading. Most of the 
concepts are simple modifications of concepts defined for probabilistic executions. 


9.3.1 Probabilistic Time-Enriched Executions 
Time-Enriched Executions 


Let M be a probabilistic timed automaton. A time-enriched execution fragment of M is a finite 
or infinite alternating sequence a = wodjw1dqw2--- where 


1. The domain of wo is [0, 0]. 
2. Each w; is a trajectory with a closed domain and each a; is an action. 


3. If a; is a visible action, then the domain of w; is [0,0], and there exists a transition 


(Istate(w;_1),P) of M such that (aj, fstate(w;)) € 2. 
A. If a; is a time-passage action, then the domain of w; is [0, a;] and Istate(wj_1) = fstate(w;). 


Denote by te-frag*(M) and te-frag(.M) the set of finite and all time-enriched execution fragments 
of M, respectively. The notation for fstate(a), Istate(a) and Itime(a) extends trivially. 

A time-enriched execution fragment a contains more information than a timed execution 
fragment since it is possible to observe what time-passage transitions are used to generate a. 

A time-enriched eztended execution fragment of M is either a time-enriched execution frag- 
ment of M or a sequence aé where a is a finite time-enriched execution fragment of W@. The 
notation for Istate(a) extends trivially. 

A finite time-enriched execution fragment a@ = wodjw,---a,w, of M and a time-enriched 
extended execution fragment a! = wl dnsiwn4i::: of M can be concatenated if Istate(a) = 
fstate(a’). In this case the concatenation is defined to be a” = upaywy-: An Wrdnd Wnts 
and is denoted by a~ a’. It is easy to see that a” is a time-enriched extended execution 
fragment of MW. A time-enriched extended execution fragment a of M is a prefix of a time- 
enriched extended execution fragment a’ of M, written a < a’, if either a = a’ or a is finite 
and there exists a time-enriched extended execution fragment a” of M such that a’ =a7 a", 
Likewise, a is a suffix of a’ if there exists a finite time-enriched execution fragment a” such 
that of = a” ~ a. Denote a by a’ra”. 
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Time-Enriched Transitions 


Let (s,P) be a combined transition of M. For each pair (a,s’) of Q, if a is a discrete action, 
then let P(a,s1) be D((a, s’)); if @ is a time-passage action, then let Pq...) be a discrete proba- 
bility distribution of Probs(trajectories( M, s,a,s')), where trajectories(M, s,a,s’) denotes the 
set of trajectories for s > s’. The pair Masyea Pl(4, $')](s, Pla,st)) is called a time-enriched 
transition of M. 

Thus, a time-enriched transition adds information to a combined transition by specifying 
what state is reached at each intermediate time. A combined transition gives just the extremes 
of a trajectory, dropping all the information about what happens in the middle. 


Probabilistic Time-Enriched Executions 


A probabilistic time-enriched execution fragment H of a timed probabilistic automaton M is a 
fully probabilistic automaton such that 


1. states(H) C te-frag*(M) 


2. for each transition tr = (q,P) of H there is a time-enriched transition tr’ = (Istate(q), P’) 
of M, called the corresponding time-enriched transition, such that P = q7~ P’. 


3. each state of H is reachable and enables one transition. 


A probabilistic time-enriched execution is a probabilistic time-enriched execution fragment 
whose start state is a start state of M. Denote by te-prfrag(M) the set of probabilistic time- 
enriched execution fragments of M, and by te-prexec(M) the set of probabilistic time-enriched 
executions of M. Also, denote by qf! the start state of a generic probabilistic time-enriched 
execution fragment H. 

As for the untimed case, there is a strong relationship between the time-enriched extended 
execution fragments of a probabilistic timed automaton and the extended executions of one of 
its probabilistic time-enriched execution fragments. Specifically, let M be a probabilistic timed 
automaton and let H be a probabilistic time-enriched execution fragment of MM. Let qo be the 
start state of H. For each extended execution a@ = qoa,q,--- of H, let 


al A qo ~ lstate( qo )ay ltraj(q1 ag ++: if a does not end in 6, (9.1) 


qo ~ lstate( go )ayltraj (qi )ag---dynltraj(qn)6 if @ = qodigi ++ Gndnd, 


where ltraj(q;) denotes the last trajectory of g;. It is immediate to observe that a| is a time- 
enriched extended execution fragment of WW. For each time-enriched extended execution frag- 


ment a of M such that go < a, 1.e., @ = qo ~ wWoayw  ---, let 
A Jods (qod1 Wy )d2(qoa1wydqw2)+-- if a does not end in 6, 
algo = von (9.2) 
gods (gods ) . -(qod1w4 . dn Wy )d if & = qoayWy ++ AnWy6. 


It is immediate to observe that aftqo is an extended execution of some probabilistic timed 
execution fragment of M@. Moreover, the following proposition holds. 
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Proposition 9.3.1 Let H be a probabilistic time-enriched execution fragment of a probabilistic 
timed automaton M. Then, for each extended execution a of H, 


(allo =a, (9.3) 
and for each time-enriched extended execution fragment a of M starting with qo, 

(atgo)| =a. (9.4) 
Events 


The probability space Py associated with a probabilistic time-enriched execution H is defined 
as for the untimed case. Thus, (24, is the set of time-enriched extended execution fragments of 
M that correspond to complete extended executions of H, i.e., 


Q, = {a| | a is a complete extended execution of H}, (9.5) 


where an extended execution a of H is complete iff either a is infinite, or a = a'6, a’ is a finite 


execution of H, and 6 € VW ate(a): For each finite time-enriched extended execution fragment 


a of M, let C# denote the cone 

CHS fal €Qy|a<a’}. (9.6) 
Let Cy be the set of cones of H. Then define Fj, to be the o-field generated by Cy, ice., 

Fiz = o(Cy). (9.7) 


Define a measure jt on Cy such that the measure jy (C”) of a cone CH is the product of the 
probabilities associated with each edge that generates a in H. Formally, let go be the start 
state of H. If a < qo, then 


wu(Cl) = 1; (9.8) 
if @ = qo ™ Woody WY ++ Wp_1Ay,Wy, then 
pa(Cl) = PR(ar,a))--- Pi, [ans dnd]; (9.9) 


where for each 2, 1 <i < n, G@ = qo woaywy +--+ wj_1ajw;; If @ = do 7 WOd{WY ++ Wy_1AnWy), 
then 


wa(C2) = PA lar,a)) PH [ans In) Pan lS], (9.10) 


where for each i, 1 <i <n, Gj = qo™ Woaywy1 +++ wj_1a;w;. Then the probability measure P;, is 
the unique measure on Fy that extends yy, and Py is the completion of Py. 


Finite Probabilistic Time-Enriched Executions, Prefixes, Conditionals, and Suffixes 


Since a probabilistic time-enriched execution is a fully probabilistic automaton, the definitions 
of finiteness, prefix, conditional and suffix of Section 4.2.6 extend directly: we just need to 
define the length of a time-enriched execution fragment a as the number of actions that occur 
ina. 
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9.3.2 Probabilistic Timed Executions 


We now define the probabilistic timed executions of a probabilistic timed automaton. We 
use probabilistic time-enriched executions to characterize those transitions that originate from 
discrete schedulers. 


Timed Transitions 


A timed transition expresses the result of choosing either an infinite trajectory or a finite 
trajectory followed by some discrete action at random. However, a timed transition should 
be the result of scheduling a collection of time-enriched transitions, so that we are guaranteed 
that it is due to a discrete scheduler. For this reason, we derive a timed transition from the 
probability distribution associated with a time-enriched probabilistic execution. The derivation 
proceeds in two steps: first all the time-passage actions are removed and the corresponding 
trajectories are concatenated; then the resulting structure is truncated at the occurrence of the 
first action. 


Removing Time-Passage Actions. Let a@ = wod,wW dgw2--- be a time-enriched execution 
fragment of a probabilistic timed automaton M. The timed execution represented by a, denoted 
by t-erec(a), is the sequence obtained from a by removing all the time-passage actions and by 
concatenating all the trajectories whose intermediate action is removed. 

Let H be a probabilistic time-enriched execution fragment of a probabilistic timed automa- 
ton M. Let 


Q = t-exec(Qy) U limits(t-exec(Qy)), (9.11) 


where limits(t-exec(Qy7)) is the set of timed executions a of M that end with an open trajectory 
and such that for each finite prefix a’ of a there is an element a” of t-exec(Qy) such that a’ < a”. 
Then, t-exec(Py) denotes the probability space completion((Q,F,P)) where F is the o-field 
generated by the cones on 2, and P is t-exec( Py). 

The reason for the definition of the sample space of t-exec( Py) is mainly technical: we 
want to establish a relationship between probabilistic time-enriched executions and probabilis- 
tic timed executions, and we want the relationship to be preserved by projection of probabilistic 
timed executions in a parallel composition context. Informally, we are interested in a distribu- 
tion over trajectories, possibly followed by an action, without keeping any information on how 
such a distribution is obtained. The elements of the sample space that end with right open 
trajectories can be affected by the way the transitions are scheduled in a probabilistic time- 
enriched execution. Moreover, these elements of 2 can create problems for parallel composition. 
Closing the sample space under limit makes such differences invisible. The reader interested in 
more details is referred to Sections 9.3.3 and 9.5, and specifically to Examples 9.3.3 and 9.5.1. 


Example 9.3.2 (What t-exec identifies) Figure 9-1 gives an example of two probabilistic 
time-enriched executions that are mapped to the same structure by t-exec(). We assume to 
have two functions w and w’ defined on the real numbers, and we denote by wy the trajectory 
w” with domain [0, d'— d] such that for each t < d’—d, w(t) = w(t —d). A similar notation is 
used for w’. a 
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Figure 9-1: Probabilistic time-enriched executions that are mapped to the same structure. 


Truncation at the First Action. Let M be a probabilistic timed automaton, and let g be 
a finite timed execution fragment of MM. For each extended timed execution fragment a of M 
such that g < a, let 


a if no action occurs in arg 


qo woayfstate (Ww) if arg = woaywy--- (9.12) 


truncate,(a) = 
Let H be a probabilistic time-enriched execution fragment of M, and let g be a prefix of 
the start state of H. Then define truncate,(t-exec(Py)) to be the probability space P where 
Q = truncate, (t-exec(Qy7)), F is the o-field generated by the cones of 2, and P is the measure 
truncate, (t-exec( Pr)). 


Timed Transitions. A timed transition of M leaving from a state s is a pair (s,P) such 
that there is a probabilistic time-enriched execution fragment H of M starting in s, and P = 
truncate ,(t-exec(PH)). 


Probabilistic Timed Executions 


A probabilistic timed execution fragment of a probabilistic timed automaton M, denoted by A, 
consists of four components. 


1. A set states( H) C t-frags(M) of states. 
2. A unique start state qi?. 
3. An action signature sig(H) = sig(M). 


4. A transition relation trans(.M) consisting of pairs (¢,P) such that there exists a timed 
transition (Istate(q),P’) of M satisfying P = ¢~ P’. Observe that, from the discussion in 
Section 3.1.5, ¢~ P’ is well defined. 


Moreover, each state of H is reachable, enables at most one transition, and enables one transition 
iff it is a finite timed execution fragment of M. A probabilistic timed execution of M is a 
probabilistic timed execution fragment of M whose start state is a start state of M. 

An execution of H is a sequence of states of H, a = qoq,---, such that for each 2, q41 € OF 
As for the untimed case, there is a strong correspondence between the timed extended execution 
fragments of a probabilistic timed execution H of M and the executions of H. Specifically, let 
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M be a probabilistic timed automaton and let H be a probabilistic timed execution fragment 
of M. Let go be the start state of H. For each execution a = qgq,--- of A, let 


o| = lima, (9.13) 


where the limit is taken under prefix ordering. It is immediate to observe that a| is a timed 
extended execution fragment of M. For each timed extended execution fragment a of M such 
that qo < a, ie., @ = qo™ woayu ---, let gi be qo~ wodiw ---a;fstate(w;), and if argo is a finite 
sequence with n discrete actions, let ¢,41 be a. Then let 


ald = gona: (9.14) 


It is immediate to observe that ago is an execution of some probabilistic timed execution 
fragment of Mf. Moreover, the following proposition holds. 


Proposition 9.3.2 Let H be a probabilistic timed execution fragment of a probabilistic timed 
automaton M. Then, for each execution a of A, 


(al)fqo =, (9.15) 
and for each timed extended execution fragment a of M starting with qo, 

(atqo)| = a. (9.16) 
Events 


The probability space Py associated with a probabilistic timed execution fragment A is defined 
similarly to the untimed case. The set 04, the set of extended timed execution fragments of 
M that correspond to complete executions of H, where an execution of H is complete iff it is 
either infinite or it leads to a state that does not enable any transition. The o-field Fj, is the 
minimum o-field that contains the class of cones of 24,;. The measure P7, is the unique measure 
that extends the measure defined on cones as follows: if a = di ~ WoGW1d2°+ + GyzWy, then 


Py[Col = Pla: PH lan) PH [Col (9.17) 


dn-1 
where for each 1 <n, qj = di ~ wod1Wy +++ dy, fstate(w;); if a = di ~ WodyWydg+-+ApW,6, then 


PylCo) = Pita): PE lanl Plo] (9.18) 


Gn-1 


where for each 2 < n, q = da ~ woa1w1-+-d,fstate(w;). Observe that although there are 
uncountably many cones in Fj,, every union of cones is expressible as a countable union of 
disjoint cones. Then, Py is the completion of P4,. 


Finite Probabilistic Timed Executions, Prefixes, Conditionals, and Suffixes 


Finiteness and prefix are defined similarly to the untimed case, and thus we do not repeat the 
definitions here. 

Conditionals and suffixes differ in a small detail concerning the start state. The reader 
should observe the similarity of these definitions to those for the untimed case. Also, observe 
that the properties of conditionals and suffixes (Propositions 9.3.3 and 9.3.4) are the same as 
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for the untimed case. This is what allows us to extend the results for the untimed case directly 
to the timed case. 

Let H be a probabilistic timed execution fragment of a probabilistic timed automaton M, 
and let ¢ be a prefix of some state of H such that q@ is a prefix of g. Then H|q is a new 
probabilistic execution fragment defined as follows: 


1. states( H|q) = {q}U {q' © states(H) | q < q’}; 


2. start(H|q) = {q}. 


3. for each state q' of H|q different from q, rile = tri. 


4, let g be the maximum state of A that is a prefix of g. Then, trifla = (q,PP|C,). 


H\q is called a conditional probabilistic timed execution fragment. We show later that H|q is a 
probabilistic timed execution. Observe that (Qy1q,FH\q, Pujq) and (QH|Cy, Fx|Cy, Px|C,) are 
the same probability space (cf. Section 3.1.8): the sample spaces are the same, the generators 
are the same, and the probability measures coincide on the generators. Thus, the following 
proposition is true. 


Proposition 9.3.3 Let H be a probabilistic timed execution fragment of a probabilistic timed 
automaton M, and let q be a prefix of a state of H such that gf! <q. Then, for each subset E 
of Qi |q; 


1. BE Frq off EF 6 Fr. 
2. If FE is an event, then Py[E] = Pu[Cy]PryqlF)- = 


Let H be a probabilistic timed execution fragment of a probabilistic timed automaton M, and 
let ¢ be a prefix of some state of H such that q is a prefix of g. Then Hoq is a new probabilistic 
execution fragment defined as follows: 


1. states( H>q) = {q'rq | q' © states( H|q)}; 
2. start( H|q) = {Istate(q)}. 
3. for each state g' of Hea, erty =tr 


Hog is called a suffix of H. It is easy to check that the probability spaces Pyp,g and Py, are 
in a one-to-one correspondence through the measurable function f : QHpg > Qy)\, such that 
for each a € Dry, f(a) = ¢~ a. The inverse of f is also measurable and associates apg with 
each timed execution a of 277;,. Thus, directly from Proposition 9.3.3, we get the following 
proposition. 


Proposition 9.3.4 Let H be a probabilistic timed execution fragment of a probabilistic timed 
automaton M, and let q be a prefix of a state of H such that gf! <q. Then, for each subset E 
of QHg: 


Ll. FEC Frog iff (q° EF) € Fu. 
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2. If E is an event, then Py|q~ E] = Py[Cq|Proql[E]. = 


We are left with showing that H|q is well defined. The proof of this apparently obvious fact is 
not simple and contains several technical details. 


Proposition 9.3.5 Let H be a probabilistic timed execution fragment of a probabilistic timed 
automaton M, and let q be a prefix of a state of H such that gf! <q. Then, H|q is a probabilistic 
timed execution fragment of M. 


Proof. We just need to verify that the transition leaving from state q in H|q is a timed 
transition. Let g be the maximum state of A that is a prefix of g. Then, from the definition 
of a timed transition, there is a probabilistic time-enriched execution fragment Hz of M such 
that pH = {7 truncate jsq1e(q(t-evec(PH,)). From the definition of tril we need to find a 
probabilistic time-enriched execution fragment H, of M such that 


(¢~ truncate tsrate(q)(t-exec(Pu,)))|Cq = q~ truncate isate(q)(t-exec( Pu, )). (9.19) 


Let gq’ be greg. From the definition of ¢, ¢’ is just one closed trajectory. Thus, if we build H, 
such that 


(t-exec(PH,))|Cq = q' ~ t-exec(Pux,), (9.20) 


then Equation 9.19 follows easily using simple properties of truncate. Thus, the rest of this 
proof is dedicated to the construction of an H, that satisfies (9.20). 

Let @1,4@2,... be an enumeration of the minimal states q” of H such that q’ < t-erec(q"). 
We distinguish two cases. 


1. For each i, t-erec(q;) = q’. 


The construction for H, in this case is carried out in the proof of Proposition 9.3.8 (cf. 
Equation 9.29). We give a forward pointer to avoid too many technical details at this 
point. 


2. There is an 7 such that q’ < t-exec(q;). 


We prove this case by reducing the problem to the previous case. That is, we build a new 
probabilistic time-enriched execution fragment Hj such that t-erec(Py,) = t-exec(Py) 


and such that the minimal states q” of H7 such that q’ < t-exec(q") satisfy q' = t-exec(q’). 


Recall first that gq’ is a trajectory whose domain is [0,d] for some d > 0. Define a 
collection of finite time-enriched execution fragments gj, q5,--- as follows: for each ¢, if 
t-exec(q;) = q' then qi = q;; otherwise, represent g; as g ~ Istate(q;)djw;, where q is 
a state of Hj, and let g be q; ~ Istate(q;)dj1wj1dj 26,24), 34;,3 where Ww; = 16,26 j3, 
t-exec( Gq; ~ Istate(q;)d;1w;1d;,2%;,2) = gq’, and the actions d; and d; 2 are chosen in such a 
way that for each i q ~ Istate(q;)dj,1;,1 is not a prefix of any of the qi’s, 7 # 7. In other 
words, we split all the g;’s in such a way that a state that corresponds to q’ is reached 
always and such that none of the states of Hz are identified. Then, 


states( H7) = {| Aid’ < ¢} (9.21) 


U (Uw ~ (q">q) | qd" € states( Hz), 4G < 1") . 
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The transition relation of Hj is obtained from the transition relation of Hz by scheduling 
the same time-enriched transitions of M as before except for the states g; where the 
intermediate transitions leading to the qi’s are scheduled. It is simple to check that Hy 
satisfies the desired properties. | 


9.3.3. Probabilistic Executions versus Probabilistic Timed Executions 


In this section we show the relationship between probabilistic executions, probabilistic time- 
enriched executions, and probabilistic timed executions. The main idea is that they all repre- 
sent the same structures with different levels of detail. We show that a probabilistic execution 
is a sampling of a probabilistic time-enriched execution, where the information given by the 
trajectories is lost. Conversely, we show that each probabilistic time-enriched execution is 
sampled by some probabilistic execution. We show that each probabilistic time-enriched exe- 
cution represents a probabilistic timed execution and that each probabilistic timed execution 
is represented by some probabilistic time-enriched execution. Essentially, a probabilistic time- 
enriched execution is a probabilistic timed execution with the additional information of what 
time-passage transitions are scheduled. Finally, we define an equivalence relation on probabilis- 
tic time-enriched executions that captures the idea of representing the same probabilistic timed 
execution. This equivalence relation will be useful for parallel composition. 


Probabilistic Executions versus Probabilistic Time-Enriched Executions 


There is a close relationship between the probabilistic executions of a probabilistic timed au- 
tomaton and its probabilistic time-enriched executions. Informally, a probabilistic time-enriched 
execution contains more information than a probabilistic execution because it associates a state 
with every real time rather than with a countable set of times. In other words, a probabilistic 
execution can be seen as a sampling of a probabilistic time-enriched execution at countably 
many points. In later chapters we will see that probabilistic executions are sufficient for the 
study of the properties of a system whenever such properties do not depend on the actual states 
that are reached at each time. For the moment we just define what it means for a probabilistic 
execution to sample a probabilistic time-enriched execution, and we show that each probabilistic 
time-enriched execution is sampled by some probabilistic execution and that each probabilistic 
execution samples some probabilistic time-enriched execution. We start by defining a func- 
tion sample that applied to a probabilistic time-enriched execution H of a probabilistic timed 
automaton M gives a probabilistic execution H’ of M, which by definition samples H. 

Let @ = wodywyd2w2--- be a time-enriched execution of a probabilistic timed automaton 
M, and let sample(a) be the sequence a’ = Istate(wo)aylstate(w, )aglstate(w2)---. Then, it is 
easy to check that a’ is an execution of MM. We say that a’ samples a. Define 


states(H') = sample(states(H)). (9.22) 


Let (gq, P) be a transition of H. Define sample on Q as follows: sample((a,q')) = (a, sample(q')), 
and sample(é) = 6. Then, define the transition sample((q,P)) to be 


sample((q,P)) = (sample(q), sample(P)). (9.23) 
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For each state q of H’, let sample—'(q) be the set of states q/ of H such that sample(q’) = 
q. Observe that all the states of sample—'(q) are incomparable under prefix. For each q’ € 
sample—'(q), let 


_sample—+(q) 4 Py[Cy] 
4 Do g!'€sample—*(q) Pr[Cq] 


Then, the transition enabled from g in H’ is defined to be 


' _sample—} 
tr = S- Py ( sample(tr/}). (9.25) 


(9.24) 


q/€sample—! (q) 


Observe the similarity of Equations (9.24) and (9.25) with the equations that the fine the 
projection of a probabilistic execution (cf. Equations (4.21) and (4.22)). 

Proposition 9.3.6 below shows that H’ is a probabilistic execution of M. We say that H’ 
samples H. Then, Proposition 9.3.7 shows that each probabilistic execution samples some 
probabilistic time-enriched execution. 


Proposition 9.3.6 For each probabilistic time-enriched execution H of a probabilistic timed 
automaton M, sample(H) is a probabilistic execution of M. 


Proof. Let H’ denote sample(H). The fact that each state of H’ is reachable can be shown 
by a simple inductive argument; the fact that each state of H’ is a finite execution fragment of 
M follows from a simple analysis of the definition of sample and of a time-enriched execution. 

We need to check that for each state g of H’ the transition enabled from q in H’ is generated 
by a combined transition of M. From (9.25), it is enough to show that for each state q’ of 
sample—'(q) the transition sample( tri} ) is generated by a combined transition of M. 

Since H is a probabilistic time-enriched execution of M, then there is a time-enriched 
transition (Istate(q’),P) of M such that PH = q'~ P. From the definition of sample and the 
definition of a time-enriched transition, (/state(q), sample(P)) is a combined transition of M, 
and sample(Pi7 ) = sample(q')~ sample(P), which means that sample(PH') = q~ sample(P). 
This is enough to conclude. | 


Proposition 9.3.7 Let H be a probabilistic execution of a probabilistic timed automaton M. 
Then there is a probabilistic time-enriched execution H' of M such that H = sample(H"'). 


Proof. We build Hf’ inductively in such a way that for each state q of H there is exactly one 
state q' of H' in sample~'(q). The start state of H’ is the same as the start state of H. 
Suppose that the transition relation of H’ is defined for each state of length at most i— 1 
and assume that for each state g of H of length at most 7 there is exactly one state q’ of H’ in 
sample—'(q). Let q be a state of H of length i and let ¢' be the state of sample—'(q). Observe 
from the definition of sample that the length of q’ is 7. Let (Istate(q),P) be the combined 
transition of M that corresponds to trf, For each pair (a,s) of 2, if a is a discrete action, 
then let Pia,s be D((a, s’)); if @ is a time-passage action, then let Piq,,1) be D(wa,s'), where 
Wa,s' € trajectories(M,s,a,s'). Let P! = Yia.syeq Pl(4, )|P(a,s). Then, (Istate(q), P’) is a time- 
enriched transition of M. Let tri be (q',q°7 P’). Then, tr is a legal transition for H’. 
Moreover, from the definition of P’, each state of pH is the sampling of exactly one state of 


PI, and, vice versa, the sample of each state of Pi is a state of PF, | 
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Probabilistic Time-Enriched Executions versus Probabilistic Timed Executions 


We define a function t-sample that, given a probabilistic time-enriched execution fragment H 
of M, builds a probabilistic timed execution H’ as follows. 


states(H') = {t-exec(qg!) U (9.26) 
{7 € Q¢.cxee(H) | ¢ contains finitely many actions} U 


{q € t-frag"(M) | liraj(q) is a [0,0]-trajectory and Aya, creorm S q'}. 


The start state of H’ is t-exec(q@! ), and for each state g of H' the transition enabled from q is 
(q, truncate, (t-exec(PH)|Cq)). 


Proposition 9.3.8 t-sample(H) is a probabilistic timed execution fragment of M. 


Proof. We need to show that for each state q of H’ that enables some transition there is 
a probabilistic time-enriched execution fragment H, of M starting from Istate(q) such that 
pH = truncate istate(q)(t-erec( Pu, )). 

Let #1, q¢2,... be an enumeration of the states q' of H such that t-exec(q’) = q, and for each 
? let p; denote Py[C,,]. Observe that, since q ends with the occurrence of a discrete action, 
for each state q” of H such that q' < t-erec(q’) there is an i such that q; < q”. Define H, as 
follows. 


states(H,) £ |) states( H>q;). (9.27) 


For each state q’ of H,, let 


vel . 
pla & Uilotestates( toa) PALCg rg lr gng PM) oon 
q ila’ Estates(Hog;) PHC g;-4'] 


Then, it is enough to prove that 
q~ t-exec(Py,) = t-exec(Py)|Cq. (9.29) 
Before proving (9.29), we show the following property: for each state q’ of Hy, 


ila! Estates \ PH|C a, 
Py [Cy] = i setetetton Fal oat (9.30) 
vr Py{Cq,| 
This follows easily by induction using Equation (9.28) for the inductive step. The denominator 
is necessary for the base case to work. 
We now turn to Equation (9.29). Consider an extended timed execution fragment a of M, 


and distinguish the following two cases. 


1. a@ does not end with an open trajectory. 


Suppose that a € Qyccece(Pyz)|C,- Then, from the definition of t-exec() and of the con- 
ditional operation, g < a@ and there is a time-enriched execution a’ of Oy such that 
t-evec(a’) = a. This means that there is a time-enriched execution a’ of Qy such that 
t-exec(a’) = a and there is a state q; of H such that q < a’. From the construction of 
H,, each prefix of a’ is a state of H,, and thus a’ € Q4.cvec(H,): The argument can be 
reversed. 
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2. a ends with an open trajectory. 


Suppose that a € Q) cxee(Py)|c,- Then, from the definition of t-exec() and of the condi- 
tional operation, g < a and for each finite prefix a’ of a there is a timed execution a” 
of t-exec(Qy) such that a’ < a”. It is sufficient to show that for each finite prefix a’ 
of a there is a timed execution aj of t-erec(Qy,) such that a’ < (q7~ a7). Consider a 
prefix a’ of a, and let a” be an element of t-exrec(Qy) such that a’ < a”. Then there is 
a time-enriched execution a!” of Qy such that a’ < terec(a’”), which means that there 
is a finite prefix a” of a!” such that a’ < texec(a’’) and q < t-erec(a’”). Let q; be 
the prefix of a’. We know that such prefix exists. Then, from the definition of H,, 
al”">qg is a state of H,, and thus there is a time-enriched execution aj of Qy, such that 
a’ < (q7 t-exec(a’,)). Moreover, t-exec(ay,) € t-exec(Py,), which is sufficient to conclude. 
The argument can be reversed. 


Finally, we need to show that Py. cxcc(Py)|Cy and Prerec(Pj,,) coincide on the cones of their sample 


Pity 
spaces. Thus, consider a finite timed execution fragment a of M. From the definition of t-ezec(), 


Pr. evec(Pu,) Cal] = S- Pr, [Cy]. (9.31) 
q'Emin({q'Estates(Hg)|a<t-exec(q')}) 


From (9.30), 


Prcvee(Py Ca] = (9.32) 
‘ q'Emin({q/Estates(Hg)|a<t-erec(q')}) di PulCqi| 
From the definition of the states of H,, (9.32) can be rewritten into 
doi Dd og'emin({q'estates(Hogi)laro<texee(qing!)}) PHC ging! 
P on _ a gi'emin({g! Estates qi)lama<t-exec(qi7q qin?" 9.33 
t-exee(Pu, )L | S, Py[Cy,] ( ) 
By simplifying the concatenations we obtain 
do g!€min({q/€states(H)|qro<t-evee(q’)}) PH[C4'] 
P Co _ gi'emin({q' Estates q7a<t-exec(q')}) q ; 9.34 
t-exee(Pu,)l | y, Pu[Cal ( ) 


From the definition of t-exec(), the definition of a conditional space, and the definition of the 


’ 
GS, 


dog! Emin ({q'Estates(H) |gra<t-erec(q’)}) Pr[Cy] 


P C,| = ——————_!—. TY>Y————. 9.35 
t-exee(P ir) |Cq| | y, Pu[Cal ( ) 

Since the right sides of Equations (9.34) and (9.35) are the same, we conclude that 
Pr. evec(Pu,) Cal] = Pr. exec (Py)|CqlCqral- (9.36) 
This completes the proof. | 


Conversely, we show that every probabilistic timed execution of M is sampled by some proba- 
bilistic time-enriched execution of 7. Let H be a probabilistic timed execution of M7. Then, 
build H’ as follows. Let Ho be a probabilistic timed execution consisting of a single state that 
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is t-sampled by gé, ie., t-sample(qa°) = gé!. Strictly speaking Ho is not a probabilistic timed 
execution because gj) ° should enable a transition in general. Suppose now that H; is defined. 
Then build H;4, be extending the transition relation of H; from all the states of H; that do 
not end in 6 and do not have any outgoing transition as follows. Consider a state g of H; that 
do not end in 6 and do not have any outgoing transition, and let q’ be the state of H such 
that t-exec(q) = q' (our construction ensures that there is always such a state since g ends with 
a [0,0]-trajectory). From the definition of a probabilistic timed execution fragment, there is 
a probabilistic time-enriched execution fragment H, of M starting from Istate(q’) such that 
PH = truncate isiate(q')(t-exec( Piz. ))- Let Hj, be obtained from H, by removing all the tran- 
sitions from states where an action has occurred and by removing all the states that become 
unreachable. Then, extend H; from gq’ with q'7~ Hi, ie., Hyped’ = Hi. 

Then the states of H’ are the union of the states of the A;’s, the start state of A’ is qe, 


and for each state q of H’, if ¢ is a state of H;, then tr = ire 


Proposition 9.3.9 t-sample(H’) = H. 


Proof. We prove that Py = t-exec(Py:). Then the equality between t-sample(H’) and 
H follows by induction after observing that t-sample(H') and H have the same start state 
and that for each state q, steps omelet) = (q, truncate,(t-exec(Py)|Cz)), and that step = 
(q, truncate,(PH|C,)). 

For the sample spaces, consider an element a of Qy. Then, by definition of Qy, there is an 
execution aga --- of A such that lim; a; = a, and such that either a is not a finite execution, 


or the last element of a ends in 6. We distinguish two cases. 


1. a is either an infinite sequence or a finite sequence agag---a, where a, ends with 6. 


From the definition of the transition relation of H’, there is a sequence of extended time- 
enriched execution fragments go,q1,... such that for each 7 a; = t-exec(qo ~ +--+ 7 Gi), 
qo~ m1 ~ +++ is an element of Qy, and t-exec(qo7 M7 --+) = a. Thus, a € 2. ccco(H. The 
converse argument is a reversal of the argument above. 


2. @ = QpQ2°--A, where a, ends with an open trajectory. 


From the definition of the transition relation of H’, there is a sequence of extended 
time-enriched execution fragments go,41,---;@n—1 such that for each 2 < n-—1 a; = 
t-erec(qo ~ +++ qj) and qo~---* q is a state of H’. Furthermore, for each finite prefix 
a’ of a there is a time-enriched execution fragment q, such that a’ < t-erec(qo~+--7 qn) 
and qo +++ Gn—1 ~ Gy is an element of Qy:. This means that for each finite prefix a’ of 
a there is an element a” of t-erec(Qq) such that a’ < a”, and thus a € Oy cree(P 1) The 
argument can be reversed. 


Consider now a cone Cy. From the definition of t-exec(), 


P1exec(H") [Cal] = S- Pr[Cq]. (9.37) 
qe min({qestates(H')|a<t-exec(q)}) 


If C, is not empty, then a = a ,---Q,, where an, = @, Ag-+-Qp_1 is an execution of H, and 
there is a aj, such that a, < af, and a,---a’, is an execution of H. We show by induction on 
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n that 
PH[Con| = S- Pr[Cy). (9.38) 
qe min({qestates(H')|a<t-exec(q)}) 


The base case is trivial since C’,, denotes the whole sample space. For the inductive case, from 
the definition of the probability of a cone, 


Pr[Can] = Pit [Coy )P [Can] (9.39) 


An-1 


From the definition of the transition relation of A, 


bo gestates(H!) |t-erec(q)=on—1 Pr(Cq) Prevec(H'eq)|Canran—t] 


PY [Con] = . 9.40 
nal yo g€states(H!)|t-exec(q)=on—1 Pr{Cq] \ ) 
where 

Pt. exec(H'eq) [Canban—| = S- PHteglCq'- (9.41) 


q'Emin({q/Estates(H'>qg)|an<t-erec(q~q')}) 


Since a,_, is a state of H, the last trajectory of a,-; has domain [0,0], and the set {q¢ € 
states(H’) | t-exec(q) = an_1} is a set of minimal states. Thus, by substituting (9.41) in (9.40), 
simplifying the numerator of (9.40), we obtain 


dig'emin({q!Estates(H")]on<t-erec(q’)}) Py [Cy] 


9.42 
yo g€states(H!)|t-exec(q)=en—1 Pr{Cq] \ ) 


Pr. evec(H'eq)|Canban—1| = 
By substituting (9.42) in (9.39), using induction and simplifying algebraically, we get (9.38). m 


Equivalent Probabilistic Time-Enriched Executions 


It is possible to define an equivalence relation on probabilistic time-enriched executions that 
captures exactly the probabilistic timed executions that they represent. 

Let H, and Hz be two probabilistic time-enriched execution fragments of a probabilistic 
timed automaton M. Then t-exec(Py,) and t-exec(Py,) are said to be equivalent, denoted by 
t-exec( Py, ) = t-exec(Py, ), iff 


1. for each timed extended execution fragment a of M that does not contain infinitely many 
discrete actions, @ € Q4 exec(P x, ) iffa€ Q4 exec(Puy)i 


2. for each finite timed extended execution fragment a of M, 
Prevee(Pu, Io = Prevee(Pu,)|Col: 


Hy, and Hy are said to be equivalent, denoted by Hy = Hy, iff t-evec(qi!') = t-evec(qi!?) and 
t-exec( Py, ) = t-exec(Py,). 


Example 9.3.3 (Two equivalent probabilistic time-enriched executions) In the defi- 
nition above we do not require the sample spaces of the given probabilistic time-enriched ex- 
ecution fragments to contain the same timed executions with infinitely many discrete actions. 
Figure 9-2 shows an example of two probabilistic time-enriched executions whose corresponding 
sample spaces differ from a timed execution with infinitely many discrete actions and such that 
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Figure 9-2: Probabilistic time-enriched executions that represent the same probabilistic timed 
execution. 


t-sample() gives the same probabilistic timed execution. The important aspect of this example 
is that in the upper probabilistic time-enriched execution the explicit time-passage actions are 
used to let 1 time unit elapse in infinitely many different ways. However, the trajectory that 
is spanned before the first occurrence of action a is always the same. Observe that the fact 
that the two probabilistic time-enriched executions of Figure 9-2 represent the same structure 
is not a consequence of the limit closure of the sample space of t-exec(), since t-exec(Qy, ) and 
t-exec(Qy,) do not differ in timed executions that end with an open trajectory. Rather, by 
analyzing this example again in the context of parallel composition we will discover the reason 
for our definition of t-exec() (cf. Example 9.5.1). = 


The rest of this section is dedicated to showing that = characterizes the probabilistic timed 
executions represented by probabilistic time-enriched executions. We do it by showing two 
results: the first result says that two equivalent probabilistic time-enriched executions describe 
the same probabilistic timed execution, and the second result says that for each probabilistic 
time-enriched execution H, P;.sampie(H) = t-exec(Py). 


Proposition 9.3.10 If t-exec( H,) = t-exec( H2), then t-sample(H,) = t-sample(H). 


Proof. Let q € states(t-sample(H,)). If q = t-exec(q,!") or ¢ € Qyceec(H,) and contains finitely 
many discrete actions, then q € states(t-sample(H2)) trivially. Thus, suppose that Itraj(q) is a 
[0,0]-trajectory and that there is aq’ € Qy..2ece(H,) Such that q < q’. Then, Py. crec(H,)[Cq] > 9, 
and, since t-exec(H,) = t-exec( Hz), Ptcrec(Hz)[Cq] > 0. Thus, there is aq” € 4 crec(H) Such 
that q < q”, which means that ¢ € states(t-sample(H2z)). The converse argument is identical. 


Consider now a state g of t-sample(H,) and t-sample( Hz). We need to show that Go? 


and ie are the same transition. From the definition of t-sample(), it is enough to show 


that truncate,(t-exec(Py, )|C,) = truncate, (t-exec( Py, )|C,). Since t-exec(Py, ) = t-exec(Pu, ), 
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a direct analysis of the definition of t-exec() shows that t-exec(Py, )|Cq = t-erec(Py,)|Cy. The 
truncation operation is independent of the elements of 2 that contains infinitely many discrete 
actions, and thus Oiruncate g(t-evec(Px, )ICq) = Ouruncateg(t-exec(P x )|Cq)° Furthermore, directly from 
the definition of =, Prruncateg(t-exec(P x, )|Cq) and Prruncateq(t-exec(P x, )|Cq) coincide on the cones, 
and thus truncate,(t-exec(Py, )|Cq) = truncate ,(t-exec( Px, )|Cy). = 


Proposition 9.3.11 Let H be a probabilistic time-enriched execution of a probabilistic timed 
automaton M. Then, P.sample(H) = t-exec(Py). 


Proof. Consider a finite timed execution a of M. We prove the proposition in three steps. 


1. For each finite timed extended execution a of M, there is a timed extended execution a’ 
of Q4.sample(H) Such that a < a’ iff there is a timed extended execution a” of 4. cree(P i) 
such that a < a”. 


Let a’ € Quy. sample(H) Such that a < a’. Then there is a complete execution qoq,--- of 
t-sample(H) such that lim;q; = a’. In particular, there is a value n such that a < qn. 
From the definition of the transition relation of t-sample(H), Pr cxec(H)|Can] > 0, and thus 
there is a timed execution a” of Q4..2ce(p,,) Such that q, < a”, which means that a < a”, 
Conversely, suppose that there is a timed execution a” of OQ; cxee(p,) Such that a < a”. Tf 
a” contains finitely many actions, then a” € Qy.sampie(H) by definition. Otherwise, there 
is a finite prefix a” of a” such that a < a” and the last trajectory of a” 
[0,0]. From the definition of t-sample(H), a’” is a state of t-sample(H), and thus there 
is a timed execution a’ of Q4.sampie(H) Such that a’” < a’, which means that a < a’, 


has domain 


2. For each timed extended execution fragment a of M that does not contain infinitely many 
discrete actions, @ € 2: sample(H) Hf a € Qycree(P yz): 


Let a be a timed extended execution of M that does not contain infinitely many discrete 
actions, and suppose that a € Qy.sample(H). If a ends with 6, then Item 1 is sufficient 
to conclude that a € Q;.ccce(py,)- If a does not end with 6, then there is a finite execu- 
tion qo%1 +++ Gn Of t-sample(H) such that g, ends with a right-open trajectory. From the 
definition of the transition relation of t-sample(H), qn, € truncate, _, (t-exec(PH)|Cq,_,)- 
Since g, ends with an open trajectory, dn € Qtexee(Py)> heey @ © Qeevec(P yz): 


Conversely, suppose that a € Qy cxec(py,)- If a ends with 6, then Item 1 is sufficient to 
conclude that a € 04. sample(H)- If a does not end with 6, then there is a finite prefix a’ of a 
such that aca’ does not contain any action, and either a’ is the start state of t-sample(H), 
or the last trajectory of a’ has domain [0,0]. Thus, from the definition of t-sample(), a’ is 
a state of t-sample(H). From the definition of truncate, a € truncate,)(t-exec(Py)|Cq'), 
and thus, from the definition of the transition relation of t-sample(H), a € om sample(H) 
Since a ends with an open trajectory, a € Qy. sample(H)- 


3. For each finite timed extended execution fragment a of M, 


Pt.sample(H)|Co] = Precxec(Py)|Cal- 


Let a be a finite timed execution. From Item 1, Co 
t-sample(H) .« 


t-sample(H) _ 0 iff an exec(Pz) _ = 0. 


Suppose that Co is not empty. Then there is an execution of t-sample(H), 
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ANA, +++ An—1QA, such that any < a < a,. From the definition of the probability of a 
cone, 


Pr. sample(H)|Ca = Pag [Cay] Po [Cas] + Pays [Cana Pan1 [Ca]. (9.43) 
From the definition of t-sample(H), for each i <n 
Po [Cais] = Ps exec(H)|Ca, (Coiarl: (9.44) 


Thus, by substituting (9.44) in (9.43) and simplifying, we obtain 


Pt.sample(H)|Ca] = Pr. ewee( Cal: (9.45) 
This completes the proof. | 
9.4 Moves 


In the non-timed framework we have introduced the notion of a weak transition to abstract 
from internal computation. Informally, a weak transition is obtained by concatenating several 
internal and external transitions so that overall the system emulates a unique transition labeled 
with at most one external action. In the timed framework, due to the presence of explicit 
time-passage actions, it may be the case that some time ¢ cannot elapse without performing 
some internal transitions in the middle. This problem becomes more evident when we extend 
the simulation relations to the timed framework (cf. Chapter 12). For this reason we introduce 
the concept of a move, which extends weak transitions and abstracts from internal transitions 
interleaved with time-passage transitions.. 

Let M is a probabilistic timed automaton, s be a state of MW, P be a discrete probability 
distribution over states of M, and a be an action of M or the value 0. If a is a visible action of 
M then we use the expression s ~> P to denote s => P; if a = 0, then we use the expression 
3 ~~ P to denote s ~ P, which is the same as s => P; if a is a time-passage action, Le., 
a = d for some d € #T, then we use the expression s <. P to denote that P is reached from s 
by means of several internal and time-passage transitions so that in each situation time d has 


elapsed. Formally, s <. P iff there is a probabilistic execution fragment H such that 
1. the start state of H is s; 
2. Pyl{aé | aé € Qy}] = 1, ie., the probability of termination in H is 1; 
3. for each ad € Oy, t-trace(a) = t-trace(a); 


4. P = Istate(é-strip(Py)), where 6-strip(Py) is the probability space P’ such that Q’ = 
{a | aé € Oy}, and for each a € 0’, P’[a] = Py[Cas]; 


The notion of a generator for a weak transition can be extended to moves in a straightforward 
way. 
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9.5 Parallel Composition 


The parallel composition operator for probabilistic timed automata is exactly the same as the 
parallel composition operator for probabilistic automata. Thus, we omit the formal definition. 
According to the definition of the transition relation of M,||M2, Mi and M2 synchronize on 
all their time-passage transitions, and thus time advances always at the same speed in M, and 
Mo. 

The definition of a projection of a probabilistic time-enriched execution is the same as the 
definition of a projection of a probabilistic execution, except that the states of a probabilistic 
time-enriched execution fragment are time-enriched execution fragments rather than ordinary 
execution fragments. Thus, we need to extend the definition of a projection to time-enriched 
execution fragments and time-enriched transitions. 

Let M be M,||Mo, and let a be a time-enriched execution of M. The projection of a onto 
M;, 1 = 1,2, is the sequence obtained from a by projecting the codomain of each trajectory 
onto M;, by removing all the actions not in acts(M;), and by concatenating all the trajectories 
whose intermediate actions are removed. It is straightforward to check that a is a time-enriched 
execution of M,;. 

Let H be a probabilistic time-enriched execution of M, and let tr = (q¢,P) be an action 
restricted transition of H such that only actions of M;, 1 = 1,2, appear in tr. Define the 
projection operator on the elements of 2 as follows: (a,q')[M; = (a,q'[M;), and 6[M; = 6. 
The projection of tr onto M;, denoted by tr[M;, is the pair (¢/M;,P[M;). 


Proposition 9.5.1 Let M = Mj,||Mo, and let H be a probabilistic time-enriched execution 
fragment of M. Then H|M, € t-prexec(M,) and H| Mo € t-prexec( M2). 


Proof. The structure of the proof is the same as the proof of Proposition 4.3.4. This time it is 
necessary to observe that for each state q of H the transition (irl | acts(M,)){Mj, is generated 
by a time-enriched transition of M;. | 


Proposition 9.5.2 Let M = Mj,||Mo, and let H be a probabilistic time-enriched execution 
fragment of M. Let H; be H| M;, i= 1,2. Let q be a state of H;. Then, 


PHiCg=  S> Pa[Cy. (9.46) 
q'Emin(q|H) 


Proof. This proof has the same structure as the proof of Proposition 4.3.5. | 


In the rest of this section we extend the results of Section 9.3.3 to account for parallel com- 
position. We show that sample commutes with projections and that the projections of two 
equivalent probabilistic time-enriched executions are equivalent. The first result guarantees 
that sample and projection are well defined for probabilistic time-enriched executions; the sec- 
ond result allows us to define indirectly a projection operator on probabilistic timed executions: 
namely, given a probabilistic timed execution H of M,||Mo, let H’ be any probabilistic time- 
enriched execution of M;,||M2 such that t-sample(H') = H. Then, H[M; is defined to be 
t-sample( H'|M;). Before proving these two results, we show why in the definition of t-exec() 
we force probabilistic time-enriched executions like those of Figure 9-1 to be mapped to the 
same structure (cf. Example 9.3.2). 
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Example 9.5.1 (Reason for the definition of t-exec) We have already seen that the prob- 
abilistic time-enriched executions of Figure 9-2 are t-samples of the same probabilistic timed 
execution. Suppose now the probabilistic time-enriched executions of Figure 9-2 to be proba- 
bilistic time-enriched executions of the parallel composition of two probabilistic timed automata 
My, and Mp, and suppose that a is an action of M2 only. By projecting the probabilistic time- 
enriched executions of Figure 9-2 onto M, we obtain two probabilistic time-enriched executions 
like those of Figure 9-1, which must denote the same probabilistic timed execution if we want 
t-sample to be preserved by the projection operation. | 


Proposition 9.5.3 Let M be M,||Mo2, and let H be a probabilistic time-enriched execution of 
M. Then, sample(H|M;) = sample(H){M;. 


Proof. Since the sampling function commutes with the projection function, sample( H|M;) 
and sample(H){M; have the same states. 

For convenience, denote sample(H) by H'. Let q¢ be one of the states of sample(H)[M;. 
Below we show that the equation for the transition leaving from q in sample(H)|M; and the 
equation for the transition leaving from q in sample(H|M;) denote the same transition. This 
is sufficient to show that sample(H)|M; and sample(H|M;) have the same transition relation. 
We use implicitly the fact that the projection onto M; distributes over the sum of transitions 
restricted to acts(M;). 

From (9.25), Proposition 4.3.2, and an algebraic simplification, the expression 


So tl PH [acts(Mi)](tr’ t acts(M;))[M; (9.47) 
q'€q|H! 
can be rewritten into 
_q|H' _sample—1(q') H 
S- S- Py Pat sample(trjn [ acts(M;))[Mi (9.48) 
q'€q|H' q'Esample—*(q') 
which becomes 
S- De egg Dar el”) sarnple( tr . acts(.M;))[M; (9.49) 
q'Esample—'(q|H’) 


after grouping the two sums. 
Denote H[|M; by H”. From (4.22), Proposition 4.3.2, and an algebraic simplification, 


S- prmele ( sample( tr") (9.50) 


q 
q’€sample—(q) 


can be rewritten into 


sample" _q! 
S- S- Py vl (D57)" PH [acts( M;)]sample( tri, t acts(.M;))[M; (9.51) 


q’€sample"(q) qq) | 
which becomes 
_sample—! _(¢"[(M,)|H 
S- Panu, al [Mi] Pi [acts(M;)]sample( tr, | acts(M;))[M; (9.52) 


q 
q'€(sample—1 (q)) |Z 
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after grouping the two sums. 

From the commutativity of sample and projection, sample~'(q|H’) = sample'(q)|H. 
Thus, in order to show that (9.49) and (9.52) denote the same transition, it is sufficient to 
show that for each state q’ of sample~'(q]H'), 


_q|H' _sample—+(sample(q")) _ _sample—1(q) _(q"[Mi)]H 
P sample(q’)Pq" _ Dy [M; Py : (9.53) 


By expanding the expressions above with their definitions, (9.53) becomes 
Pu |C sampte(q) | PalCq"| 
(soy min (q]H) Pr [Co] og esample—}(sample(q’)) Py|[Cq]) 
Py [Corp | Pal Cq"] 
(sa esample—} (q) Pr (Co)) (are min((q" [M;)|H) Py|Cq)) 


By simplifying common subexpressions, using Proposition 4.3.5, and observing that 


Pr (C sample(q')] = S- Pr{Cp], (9.55) 


g'Esample—1(sample(q")) 


(9.54) 


(we have verified properties like (9.55) several times) Equation (9.54) becomes 
S> PH[Cr] = S- Pro [Cy], (9.56) 
q'Emin(q|H’) q’Csample—*(q) 
which can be shown as follows: 
Yo PalCe] 
q’Emin(q|H’) 
= S- S- Pr[Cq] 
qemin(q| A’) q" sample! (97) 
= S- Pr{Cq] 
q!Emin(sample—1(q]H’)) 
= S- Pr[Cq"] 
q!Emin((sample—! (q))|#) 


= > > Py[C qr] 


q’Csample—1(q) a Emin(9’]H) 


= Ply, 


q’Csample—"(q) 


where the first step follows from (9.55), the second and fourth steps follow from grouping and 
ungrouping sums, the third step follows from the commutativity of sample and projection, and 
the fifth step follows from Proposition 4.3.5. | 


Proposition 9.5.4 Let H, and Hz be two probabilistic time-enriched executions of M,||M2. If 
Ay = FT, then Ay [.M; = Hy [M;, t= 1,2. 
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Proof. We show first that t-exec(Py, ru; ) and t-exec(Py, ru; ) assign the same probabilities 
to the same cones; then we show that the sample spaces of t-exec(Py, ru; ) and t-exec(Py, (Mi, ) 
satisfy the condition for =. This part of the proof relies on the way we have defined the sample 
spaces of the objects produced by t-exec(). For the cones, we show that for each finite timed 
extended execution a of M;, 


Prevee(Pu, ru, [Col = S- Pr. exec(H,)Co’]- (9.57) 
al€min({a'€t-frag 5 (M1 ||Mz2)|o=a'[M;}) 


and 


Preevee(Pay ru, [Col = S- Pr. exec(Hy)(Co']- (9.58) 
al€min({a'€t-frag 5 (Mi ||Mz2)|o=a'[M;}) 
Then, since Hy; = H2, we conclude that the right sides of (9.57) and (9.58) are equal, and thus, 
Hy |M; = H2[M;. We prove only (9.57); the proof for (9.58) is symmetric. From the definition 
of t-exec(), 


Px coce(P x, pur,)|Cal = S- Pr, tulCd)- (9.59) 
qgeémin({qestates( Hy [M;)|a<t-exec(q)}) 


From (4.31), 


Pe cvce(Pry txs,)[Cal = S- ( Ss” Pa, ca] (9.60) 
gemin({qestates(Hi[Mi)|a<t-ewec(q)}) \g’€min(q]H1) 

Consider a state g of min({q © states(H,[M;) | a < t-exec(q)}) and a state q' of min(q|H1). 
Then, from the definition of t-exec(), there is at least one a’ € t-frag$(M,||M2) such that 
a = a’/M; and qd’ € min({q' © states(H,) | a’ < t-exec(q')}). Moreover, there is exactly 
one minimum a’. Conversely, consider one a’ € min({a’ € t-frag3(Mi||M2) | a = a’[M;}), 
and consider a state q' of min({q’ € states(H,) | a’ < t-exec(q')}). Let g = q/[M;. Then, 
q’ € min(q|H,) and q is a state of min({q € states( H,[M;) | a < t-exec(q)}). Thus, from (9.60) 
we obtain (9.57). 

We now move to the sample spaces. Let a be an element of Oy cvec(Py, ru,) that does not 
contain infinitely many discrete actions. If a ends with 6, then a is trivially an element of 
Q4 exvee(Pu, ru,) Since Pr ecec(Px, ru, [Col = P1.evec(Puyu,) Col > 0. Otherwise, a ends with an 


open trajectory. Then, from the definition of Qy cree , for each finite prefix a’ of a there 


Pu, [u,) 
is an element a1 of t-exec(Qy, ru; ) such that a’ < ay. it is enough to show that for each finite 
prefix a’ of a there is also an element ag of t-exec(Qy,;34,) such that a’ < ay. 

Let a’ be a finite prefix of a such that there is an element a1 of t-exec(Qy, py, ) such that 
a” < a;. Thus, there is a time-enriched execution a4 of Qy, 517, such that a’ < t-erec(a‘). 
This means that there is a state q, of H,/M; such that a’ < t-erec(q,). From the definition 
of projection, there is a state g of H, such that a’ < t-erec(q,[M;), and thus there is a timed 
execution aff of t-exec(Qy,) such that a’ < (a{/M;). Consider a finite prefix ai’ of aif such 
that a’ < (af/|M;). Then, P.. cvee(Py, ICam'| > 0. Since Hy = Ho, Prevee(Py,)[Cam] > 0, which 
means that there is a timed execution af of OQ 4 exee(P Hy) such that a’ < (a¥[M;). Thus, there 
is a state qi of Hz such that a’ < t-erec(q,[M;), and from the definition of projection, there 
is a state gq. of H2[M; such that a’ < t-exec(q.). This implies that there is an element a‘ of 


t-exec(Qy,;4,) such that a’ < a5, which is sufficient to conclude. a 
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9.6 Discussion 


To our knowledge, no general probabilistic models with dense time have been proposed except 
for the automata of Courcoubetis, Alur and Dill [ACD91la, ACD91b]. In our model no prob- 
ability distributions over passage of time are allowed within a probabilistic timed automaton; 
time can elapse probabilistically only within a probabilistic timed execution, and the associated 
probability distributions can be only discrete. We have chosen to define the timed model with 
such a restriction so that all the theory for the untimed model carries over. 

Further work should investigate on the extension of our model to non-discrete probability 
distributions. A starting point could be the study of restricted forms of non-discrete distri- 
butions as it is done by Courcoubetis, Alur and Dill in [ACD91la, ACD91b]. Useful ideas can 
come from the work on stochastic process algebras of Gétz, Herzog and Rettelbach [GHR93], 
Hillston [Hil94], and Bernardo, Donatiello and Gorrieri [BDG94]. 
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Chapter 10 


Direct Verication Time 
Complexity 


Part of this chapter is based on joint work with Anna Pogosyants and Isaac Saias; some of the 
ideas have been influenced by discussion with Lenore Zuck. The verification of the randomized 
dining philosophers algorithm of Lehmann and Rabin (Section 10.6) is based on joint work 
with Nancy Lynch and Isaac Saias [LS$94]; the verification of the randomized algorithm for 
agreement of Ben-Or (Section 10.8) is joint work with Anna Pogosyants and is a formalization 
of a proof that appears in the book on distributed algorithms of Nancy Lynch [Lyn95]. Close 
interaction with Anna Pogosyants lead us to the idea of the abstract complexity measures of 
Section 10.7. 


10.1 General Considerations About Time 


The direct analysis of a probabilistic timed automaton is carried out exactly in the same way 
as for untimed probabilistic automata. Thus, probabilistic statements and progress statements 
can be generalized directly, and the coin lemmas can be applied without any modification. 

In this chapter we concentrate more on topics that are specific to the presence of time. In 
particular, it is now possible to enrich the notation for progress statements and verify some of 
the real-time properties of a probabilistic timed automaton. We extend the progress statements 
of Chapter 5 by adding a time parameter ¢: the expression U = U’ means that, starting from 


a state of U, astate of U’ is reached within time ¢ with probability at least p. Based on the new 
timed progress statements we show how to derive upper bounds on the worst expected time for 
progress. 

We generalize the method for time complexity analysis to more abstract complexity mea- 
sures. Then, rather than studying the expected time for progress, we study the expected 
abstract complexity for progress. We use abstract complexity to derive an upper bound on the 
worst expected time for decision of the randomized algorithm for agreement of Ben-Or that we 
presented in Chapter 5. Specifically, we show that under some conditions on the scheduling 
policy, each non-faulty process completes its i** stage within some upper bound, and we show 
an upper bound on the expected number of stages that are necessary to reach agreement. In 
this case the abstract complexity is the number of stages. A direct analysis of the expected time 
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for success in Ben-Or’s algorithm would not be as easy since there is no useful upper bound on 
the time it takes to a process to move from a stage to the next stage. 

Sections 10.2, 10.3, and 10.4 simply extend the definitions of Chapter 5 to the timed case; 
Section 10.5 shows how to derive upper bounds on the worst expected time for progress given 
a timed progress statement, and Section 10.7 shows how to derive upper bounds on the worst 
expected abstract complexity for progress given a timed progress statement with abstract com- 
plexity; Sections 10.6 and 10.8 present examples of application by proving that the randomized 
dining philosophers algorithm of Lehmann and Rabin guarantees progress in expected constant 
time and that the randomized agreement algorithm of Ben-Or guarantees agreement in expected 
exponential time. 


10.2 Adversaries 


An adversary for a probabilistic timed automaton M is a function A that takes a finite timed 
execution fragment a of M and returns a timed transition of M that leaves from Istate(a). 
Formally, 


A : t-frag*(M) — t-trans(M) 


such that if A(a) = (s,P), then s = Istate(a). Moreover, an adversary satisfies the following 
consistency condition: if A(a) = (s,P), then for each prefix a’ of some element a” of Q, 
A(a~ a’) = (Istate(a’), Poa’). Informally, consistency says that an adversary does not change 
its mind during a timed transition. 

An adversary is deterministic if it returns either deterministic timed transitions of M or 
pairs of the form (s, D(sé)), ie., the next timed transition is chosen deterministically. Denote 
the set of adversaries and deterministic adversaries for a probabilistic timed automaton M by 
Advs(M) and DAdvs(M), respectively. 

The definitions of an adversary schema and of the result of the interaction between an adver- 
sary and a probabilistic timed automaton is the same as for the untimed case (cf. Section 5.2), 
and thus we do not repeat them here. 

To guarantee that our adversaries are well defined, we need to prove the following lemma. 


Lemma 10.2.1 [f(s,P) is a timed transition of a probabilistic timed automaton M, then for 
each prefix a’ of some element a” of Q, (Istate(a’), Pra’) is a timed transition of M. 


Proof. This is proved already in Proposition 9.3.5. | 


10.3. Event Schemas 


As for the untimed case we need a mechanism to associate an event with each probabilistic 
timed execution fragment of a probabilistic timed automaton. Thus, an event schema is a 
function e that associates an event of the space Py with each probabilistic timed execution 
fragment H of M. The notion of finite satisfiability extends directly from the untimed case. 
Observe that, although in Py there can be uncountably many cones, each finitely satisfiable 
event can be expressed as the union of countably many disjoint cones. Furthermore, every 
uncountable family of cones contains at least two cones that are not disjoint. 
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The definition of a timed probabilistic statement extends directly from the untimed case, and 
similarly the definition of the concatenation of two event schemas extends directly. Therefore, 
we omit the definitions, which are identical to those of Chapter 5. 


Proposition 10.3.1 The concatenation of two event schemas is an event schema. That is, if 
€ = €1 OCones €2, then € is an event schema. 


Proof. Consider a probabilistic timed execution fragment H. From Proposition 9.3.3 each set 
€2(H|q) is an event of Fy. From the closure of a o-field under countable union, e(/) is an 
event of Fy. | 


Proposition 10.3.2 Py[e1 ocones €2(H4)] = Vyecones(H) PHICal Paqleo( 4 14)I- 


Proof. Since Cones(H) represents a collection of disjoint cones, from (5.13) we obtain 


Pyles Cones €2(H)] = S- Pyie2( H|q)). (10.1) 
q€ Cones(H) 


From Proposition 9.3.3, for each ¢ € Cones(H) 
Paleo(H|q)] = Pu[Cq]Puqlea(#19)]- (10.2) 
By substituting (10.2) in (10.1) we obtain the desired result. = 


Now it is possible to prove a concatenation property similar to the one for the untimed case. 


Proposition 10.3.3 Consider a probabilistic timed automaton M. Let 
1. Pr4dvs,o(€1) R pr and, 
2. for each A € Advs, q € O, let Pr jays Cones(prevec(M,A,q))(€2) R P2- 
Then, Pr advs,o(€1 Cones €2) R Pipa: 


Proof. Consider an adversary A € Advs and any finite timed execution fragment g € 0. Let 
H = prexec(M,A,q). From Proposition 10.3.2, 


Priet © Cones e2(H)| = S- Py[Cq | Pag le2( 47). (10.3) 
q'€ Cones(H) 


Consider an element gq’ of Cones(#H). It is a simple inductive argument to show that 
H\q' = prezec( M, A, q’), (10.4) 


where we use consistency for the base case. Thus, from our second hypothesis, 


Pryjqile2( 1 |q)] R pr. (10.5) 
By substituting (10.5) in (10.3), we obtain 
Pret Cones €2(H)] R po S- Py[Cy']. (10.6) 


q'€ Cones(e1(H)) 
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By using the fact that Cones(H) is a characterization of e;(H) as a disjoint union of cones, 
Equation (10.6) can be rewritten into 


Pule1 OCones €2(1)| R poPuler(H)]. (10.7) 
From the first hypothesis, Py[e1()] R pi; therefore, from Proposition 5.4.1, 

Prlei ©Cones €2(H)] R pipe. (10.8) 
This completes the proof. | 


10.4 Timed Progress Statements 


As a special case of a probabilistic statement for the timed case we can add some features 
to the notation XY yrAdvs X’. In particular we define a timed progress statement to assert 


that starting from a set of states U some other state of a set U’ is reached within time t with 


probability at least p. Such a statement, which we denote by U —Advs U', or by U = U' if 


Advs is clear from the context, is expressed by the probabilistic statement Pradus,.u(eut) > p, 
where the event schema ez, applied to a timed probabilistic execution fragment H returns the 
set of timed executions a of Qy where a state from U’ is reached within time t in avgé!. Such 
a set can be expressed as a union of cones, and therefore it is an event. 

Similarly, the progress statements involving actions can be generalized to the timed frame- 
work. Thus, V Ades V’ is the probabilistic statement PrAdus,Oy (evs) > p, where Oy vy: is 
the set of finite timed execution fragments of Mf where an action from V occurs and no action 
from V’ occurs after the last occurrence of an action from V, and the event schema ey: ; applied 
to a timed probabilistic execution fragment H returns the set of timed executions a of Qy such 
that an action from V occurs in avg)! within time t. 

In order to generalize the concatenation theorem for progress statements, we need to extend 
the definition of a finite-history-insensitive adversary schema. Thus, an adversary schema Advs 
is finite-history-insensitive iff for each adversary A of Advs and each finite timed execution 
fragment a of M there is an adversary A’ of Advs such that for each timed execution fragment 
a’ such that a < a’, A(a’) = A’(a’>a). Then, the following theorem is shown in the same way 
as for the untimed case. 


Theorem 10.4.1 Let Advs be finite-history-insensitive. If X des X' and X! Po sdes xX", 
1 2 


ty +t 
then X Ate ius XxX", | 
P1P2 


10.5 Time Complexity 


In this section we show how to study the time complexity of a randomized distributed algorithm. 
We start by defining how to compute a worst expected time, and then we show how it is possible 
to derive upper bounds on the worst expected running time of an algorithm based on timed 
progress statements. 
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10.5.1 Expected Time of Success 


Let e bea finitely satisfiable event schema and suppose that Py[e(H)| = 1,i.e., that the property 
described by e is satisfied in H with probability 1. Let Cones(H) be a characterization of e(/) 
as a disjoint union of cones, where each element of Cones(#) identifies the first point along 
a timed execution where the property denoted by e is satisfied. Then, we can compute the 
expected time to satisfy the property identified by e as 


S- Py[Cq](ttime(qogg’)). (10.9) 
q€ Cones(H) 


In general, if € is a finitely satisfiable event-schema and Cones( H ) identifies the first point along 
a timed execution where the property identified by e is satisfied, then for each probabilistic timed 
execution fragment H of M we define F'y[e], the expected time to satisfy e in H, as follows. 


Eyle) ( Dyeconea nr) PulCa|(ltime(qoad!)) if Pyle] = 1 410.10) 


ove) otherwise. 


Then, the question is the following: are there easy ways to compute upper bounds on the 
expected time for success in a randomized algorithm without computing explicitly (10.10)? We 
give a positive answer to this question. 


10.5.2 From Timed Progress Statements to Expected Times 


Timed progress statements can be used to analyze the time complexity of a randomized algo- 
rithm. The main idea for the analysis is expressed by Proposition 10.5.1. Suppose that we 
know the following: 


U Ades U (10.11) 
U => (U Unless U"). 


Then, if Advs is finite-history-insensitive and 56 ¢ Q4(s) for each A € Advs and each s € U, 
we know from Proposition 5.5.6 that U —yrAdvs U’. Let e be a finitely satisfiable event schema, 


and let Cones express the points of satisfaction of e. Suppose that for each probabilistic timed 
execution fragment H and each state q of H, if there is no prefix q’ of g such that q' € Cones(H), 
then e(Hpq) = e(H)>g and Cones(Hrq) = Cones(H)rg (e.g., e can express the property of 
reaching some state in a set U", or the property of performing some action). Let 

Ev, Advs[€] £ SUP seU, AE Advs "’prevee(M,A,s) |] (10.12) 
Then the following property is valid. 
Proposition 10.5.1 

Ev,Advsle] < t + pEut advsle] + (1 — p) Ev, advs[e]- (10.13) 
Proof. We prove (10.13) by distinguishing four cases. 


1. Eu: aavsle] 2 Eu,Adsle]. 
In this case (10.13) is satisfied trivially. 
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2. Ev,Adusle] = 00 and p < 1. 
Also in this case (10.13) is satisfied trivially. 


3. Ev,Adusle] = 00 and p= 1. 


We show that Fu: adys[e] = 00, which is enough to satisfy (10.13). Suppose by contradic- 
tion that Ku 4sdys[e] < co. Then we distinguish the following cases. 


(a) There is an adversary A of Advs and a state s of U such that 
Porevee(M,A,s) e(prexec( M, A, s))] < 1. 

(b) It is not the case that there is an adversary A of Advs and a state s of U such that 
Porexec(M,A,s) e(prexec( M, A, s))] < 1. 


For Case (a), let Conesy be the function that expresses the points of satisfaction of ez, 
and let H be prevec( M,.A,s), where P. (u,A,s)le(prezec( M, A, s))] < 1. Then, 


prexec 


Pale > SD PulCy)Prrogl (Hoa), (10.14) 
q€ Cones (H) 


i.e., the probability of satisfying e is not smaller than the probability of reaching U’ and 
then from there satisfying e. From the finite-history-insensitivity of Advs, for each state q¢ 
of Conesy:(H) there is an adversary A’ of Advs such that Hog = prexec( M, A’, Istate(q)), 
and thus, since Ey: adus[€] < 00, PHpg(e(Heq)) = 1. By substituting this result in (10.14), 
we get 


Pyie( H)] > S> Pa[Ci].- (10.15) 
q€ Cones (H) 


Since p = 1, the right side of (10.15) is equal to 1, ie., Py[e(H)] > 1, a contradiction. 


For Case (b), let Conesy be a function that expresses the points of satisfaction of ev, 
and, for each d > 0, let Conesg be a function that expresses the event of reaching time 
das a union of disjoint cones. From the definition of a probabilistic timed execution, 
we know that Cones exists and that for each probabilistic timed execution fragment H 
and each q € Conesa(H), Itime(qrgi!) = d. Let H be prexec(M,.A,s). From (10.10) the 
expected time for success for e is 


Eyle] = S- Py[Cy]ltime(qoge ). (10.16) 
q€ Cones(H) 


Let € be an arbitrary positive number. Let 01 be the set of elements q of Conesy:(H) 
such that Itime(qrgé!) < t+e, and let Hy» be the set of elements ¢ of Cones;4,(H) that do 
not have any prefix in 01. Since Pyley(H)] = 1, then Py[Ugee,ue,Cq] = 1. Moreover, 
by hypothesis, Pr[UjeCones(H)Cq] = 1. Thus, observe that each element of Cones(H) has 
either a proper prefix or a suffix in 0; U Og. In fact, if there is an element q of Cones(H) 
that has no prefix nor suffix in @; UQ2, then the cone C, would not be part of Ujseo, ue, Cy, 
contradicting the hypothesis that Py[Uge Cones(H)C 4] = 1. Similarly, we can show that 
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for each element qg of 01 U ©2 has either a prefix or a proper suffix in Cones(H). Thus, 
Cones(H) can be partitioned into two sets O? and O* of elements that have a proper 
prefix and a suffix, respectively, in 0, UO, and ©; U Og can be partitioned into two sets 
Ofs and Oj » of elements that have a prefix and a proper suffix, respectively, in Cones( I). 
Based on these observations, the right side of Equation( 10.16) can be rewritten into 


S- S- Py[Cq'|PHeg'[Cqrq'|(Itime(q'e gd) + ltime(qrdq’)) (10.17) 
qEOP gEOF pq’ <q 


+ S- S- Py[Cq] PHoglCy'oq] ltime(qogg’ ) 
q€O* EO? o|\a<q" 


Observe that for each gq € O°, diacor ,|q<q! PHog[Cqeq] = 1, and observe that for each 


¢ € OF 2, Vqeor|q'<q PHog [Cqrq'] = 1. By exchanging the sums in (10.17) and using some 
simple algebraic manipulations, we obtain 


S- Pr{Cq'] [ninco S- Pel CtinCet) (10.18) 


gO} » gE OP |q'<q 


+ S- S- Py[Cq] PHoglCy'oq] ltime(qogg’ ) 
q' EOF» FEO®|g<4! 


In the first summand, since from the properties of e for each q’ € Of, e(Hpq’) = 
e(H )og’, the subexpression )7copjq'<q [time(qrg') Preg'[Cgng'] denotes Eyzpq/[e]. In the 
second summand, observe that for each gq’ € Ofs there is exactly one element g of 0° 
such that ¢ < q’. Moreover, Pa[C]PH»q|Cq'eq] = PaH[Cq/]. Thus, from (10.18) we obtain 


Entel < | SD PulCyl(ttime(q'egl!) + Enel) (10.19) 
gO} » 


+ S- Py[C,'|ltime(q'egg ) 
gEO7» 


By repartitioning OF 5 U Ofs into 0, and Og, and by observing that for each element q 
of O, Itime(qrgé) < t+, and for each element ¢ of O2 Itime(qrgi!) = t + €, (10.19) can 
be rewritten into 


Enll<(t+Q]  S> PalC nell) | +] S2 PalCJEnqle | (10.20) 
qeOy N01 qeOf .NO, 


+ S> Par[Cq]E nae] | + So Pa[Cq]Ev,aavslel | 5 
qEQ} .NO2 qeOY M2 
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where we have added F7,,[e] in the upper right summand and Fy, 4dys[e] in the lower 
right summand. Since Advs is finite history insensitive, for each g € 0, U Og there is an 
adversary A’ of Advs such that (Heq) = prexec( M, A, Istate(q)). Thus, (10.20) can be 
rewritten into 


qeO1 qEO2 


Ene] < (t+ 6) (> Py[Cq\ Eur, aavsle ») + (=: Py[Cq)Eu,Advsle ) (10.21) 


where we have used U > (U Unless U') to say that the last states of the elements of O02 
are in U. Observe that )°,¢o, Pa[C] is Puleuy4(4)], which is 1 by hypothesis. Since by 
hypothesis Ey 4dvs[e] < 00, from (10.21) we derive that Eu,adus[e] < 00, a contradiction. 


. Ev, adusle] < 0, Eur adsle] < 00, and Ey aasle] < Eur saslel- 

Let A be an adversary of Advs and s be a state of U. Let H be prevec(M,A,s). Let € 
be any positive real number. Equation (10.21) can be derived also in this case using the 
same identical argument as before. Since we have assumed that Ey: adus[e] < Eu,Adusle], 
the lowest possible value of the right side of (10.21) occurs by giving U’ the lowest possible 
probability, which is p. Thus, (10.21) becomes 


Enle] < (t+ ©€)pEur,savsle] + 1 — p) Eu, aavsle]. (10.22) 


Since Equation (10.22) is valid for any adversary Advs and any state of U, we obtain 
timed execution fragment 


Ev, Advsle] < (t+ ©)pEut,advsle] + (1 — p) Eu, advsle]- (10.23) 


Since Equation (10.23) is valid for every «, Equation (10.23) is valid also for the infimum 
of the values that € can have, i.e., 0, and thus, 


Ev,Advsle] < t + pEu,advsle] + 1 — p) Ev, advsle]- (10.24) 


This completes the proof. | 


Example 10.5.1 (From timed progress to expected time) As a simple example of ap- 
plication of Proposition 10.5.1, suppose that e expresses the property of reaching U’. Then, we 
know by definition that Lu: ads[e] = 0. By applying Equation (10.13), we obtain Ey 4dvs[e] < 


— p)Ev,adusle], which gives Fy 4davs[e] < t/p, ie., the expected time to reach U' from U 


is at most t/p. Informally speaking, we can view the process of reaching U’ as a sequence of 
Bernoulli trials, each one performed every ¢t time units. At time ¢, with probability p we have 
reached U’, and with probability (1 — p) we are still in U, and thus we apply the same exper- 
iment again. The expected number of rounds of such a process is 1/p, and thus the expected 
time for success is t/p. Suppose now that we know the following, 


Up —bsars U1 Up > (Uo Unless U1) 
bb (10.25) 


Uy Po sdes Uy U, => (U, Unless U2), 
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and suppose that e expresses the property of reaching U2. Then, we know that Eu,,4avs[e] = 0. 
By applying Proposition 10.5.1, we obtain 


Eup, Advs[€] < t1 + pi Ev, Advsle€] + 1 — pr) £05, Advslé] (10.26) 
Ev, ,Advs[€] < t2 + (1 — po) Eu, Adv [e]. 


ao 


From simple algebraic manipulations (10.26) becomes 


Eup, Advs[€] < ti /pi + Ev, Advs [le] 
’ ’ 10.27 
Lu, ,Advs € < ty/po, ( ) 


and thus, after substituting the second inequality in the first inequality, 


Eu, Adusl€] < t1/pi + t2/ po 
10.28 
Lu, ,Advs el < tz /po. ( ) 


Suppose now that in addition to (10.25) we know that 


t3 

Up = (Uo Unless U2), 

which is possible if U, C Up U Ug. Then, from Proposition 10.5.1 we get 
Eup, Adus [e] < ts/ps, (10.30) 

which added to (10.28) gives 

Lup ,Advs [e] < min(ty/pr + to/p2, ts/p3) (10 31) 

Eu, Adusl€] < t2/pe- ; 
Therefore, more information may give us the possibility to prove better bounds. | 


Proposition 10.5.1 can be proved also for timed progress statements that involve sets of actions 
rather than sets of states. Let V,V’ denote two sets of actions, and let Advs be an adversary 
schema. Suppose that 


Vv —Advs Vv’ (10.32) 


Let e be a finitely satisfiable event schema, and let Cones express the points of satisfaction of 
e. Suppose that for each probabilistic timed execution fragment H and each state q of HA, if 
there is no prefix gq’ of ¢g such that q' € Cones(H), then e( Hog) = e(H)>g and Cones( Hrg) = 
Cones(H)og. Let Evv'advsle] denote SUP gO yy ,ACAdvs E’prevec(M,A,q) [€l- Let Oy denote the 
set of finite execution fragments of M whose last action is in V’, and let Ey? 4ays[e] denote 
SUP ge0y,1, AE Adve” prevee(M,A,q)LE]+ Suppose that q’6 ¢ Q 4(q) for each q’, each A € Advs and each 
q € Ovyy. Then the following proposition is valid. 


Proposition 10.5.2 
1. Ev.viAavsle] < t+ pEv'Advsle] + 1 — p)Ev,v'Advsle], and 
2. for each set of actions V", Ey' aasle] < Evive Ade]: 


Proof. The proof of the first item follows the lines of the proof of Proposition 10.5.1; the proof 
of the second item follows from the fact that Oy: C Oy yu. | 
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10.6 Example: Randomized Dining Philosophers 


To illustrate the use of timed progress statements for the analysis of an algorithm, we reconsider 
the randomized dining philosophers algorithm of Lehmann and Rabin, and we show that, under 
the condition that each process has a minimum speed, progress is guaranteed within expected 
constant time. First, we show how to add time to the probabilistic automaton that describes the 
algorithm; then, we add time limitations to the progress statements that we used in Section 6.3.3 
and we derive the upper bound on the expected time for progress; finally we repeat the low 
level proof observing that the coin lemmas are applied in the same way as for the untimed case. 


10.6.1 Representation of the Algorithm 


The probabilistic timed automaton that represent the Algorithm of Lehmann and Rabin can be 
obtained directly from the probabilistic automaton of Section 6.3.2 by adding arbitrary self-loop 
time-passage transition from each state (same as the patient construction of Example 9.2.1). 
Then, in order to enforce a lower bound on the speed of each process, we impose some limitations 
on the adversaries that act on M. For convenience, but without loss of generality, we assume 
that from any point each process in its trying or exit region performs one transition within time 
1. Thus, the adversary schema that we use on M is the set of adversaries A for M such that 
for each finite timed execution fragment a of M, 


1. Porevee(M,A,a) frag (M1) = 1, and 


2. for each element a’ of Qorerec(M,A,a) there is no pair of prefixes ay < ag of a’>a and no 
process ? such that process 7 is in its trying or exit region in Istate(a1), Itime(az>a,) > 1, 
and process ? does not perform any discrete transition in agra. 


We call this adversary schema Unil-Time. 


Remark 10.6.1 Observe that in Condition 1 we require the probability of the admissible 
executions to be 1 rather than requiring the sample space to contain only admissible executions. 
The reason for using probabilities is technical and is due to the fact that the sample space of a 
probabilistic timed executions always contains Zeno timed executions, even though they occur 
with probability 0. From the practical point of view all the Zeno timed executions can be 
ignored. 

In other words, it is not necessary to know the intricacies of the definition of a probabilistic 
timed executions since they are used only to guarantee that the events of interest are measurable. 
From the point of view of verifying the correctness of a randomized distributed algorithm, as 
long as Zeno timed executions occur only with probability 0, it is possible to think that Zeno 
timed executions do not occur at all. | 


Remark 10.6.2 (Alternative approach) Another alternative approach to modeling the al- 
gorithm of Lehmann and Rabin, which we do not use here, is to augment the probabilistic 
automaton of Section 6.3.2 with an upper bound for each process 2 to the time by which pro- 
cess ? must perform a transition, and to allow a time-passage transition only when no process 
goes beyond its upper bound. Of course the upper bounds need to be updated opportunely 
within a transition. In this case the condition imposed on an adversary would be just that time 
advances unboundedly with probability 1. | 
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10.6.2 The High Level Proof 


The high level proof consists of the same progress statements that we used in Section 6.3.3 
together with a time bound. Specifically, we use the following timed progress statements. 


T — RT UC (Proposition 10.6.3), 

RT . FUGUP (Proposition 10.6.15), 

F ar GUP (Proposition 10.6.14), 

G ai P (Proposition 10.6.11), 

P — C (Proposition 10.6.1). 
By combining the statements above by means of Proposition 5.5.3 and Theorem 10.4.1 we 
obtain 

Te. (10.33) 


Observing that if some process is in the trying region then some process is in the trying region 
unless some process gets to the critical region, we apply Proposition 10.5.1 and we obtain that 
the expected time to reach C from RT is at most 104, i.e., the algorithm of Lehmann and Rabin 
guarantees progress within expected constant time. 


10.6.3. The Low Level Proof 


We now prove the timed progress statements of Section 10.6.2. The proofs are exactly the same 
as the proofs given in Section 6.3.4 with the difference that in this case we consider also time 
bounds and we consider only admissible timed execution fragments since we know that they 
occur with probability 1. 


Proposition 10.6.1 Jf some process is in P, then some process enters C’ within time 1, i.e., 
1 


Proof. Let i be the process in P. Then, from the definition of Unit-Time, process 7 is scheduled 
within time 1, and enters C. | 


Lemma 10.6.2 If some process is in its Exit region, then it will enter R within time 3. 


Proof. The process needs to perform two transitions to relinquish its two resources, and then 
one transition to send a rem message to the user. Every adversary of Unit-Time guarantees 
that those three transitions are performed within time 3. | 


Proposition 10.6.3 7 —.RTUC. 
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Proof. From Lemma 6.3.2, every process that begins in Pr or Eg relinquishes its resources 
within time 2. If no process begins in C or enters C' in the meantime, then the state reached 
at this point is a state of RT; otherwise, the starting state or the state reached when the first 
process enters C’ is a state of C. | 


We now turn to the proof of G ai P. The following lemmas form a detailed cases analysis 


of the different situations that can arise in states of G. Informally, each lemma shows that a 
specific coin event is a sub-event of the properties of reaching some other state. Here we do not 
repeat the proof of Lemma 6.3.4 since it does not depend on timing issues. 


Lemma 10.6.4 


1. Let X;-, € {Ep, R, F} and X; = W. If FIRST(flip,;_,,left), then, within time 1, 
either X;_1 = P or X;=S. 


2. Let X;-1 = D and X; = W. If FIRST (flip,_,,left), then, within time 2, either 
X;-1 = Por X;= 5S. 


3. Let X;-1 = S and X; = W. If FIRST(flip,;_,,left), then, within time 3, either X;_1 = 
Por X;=5S. 


4. Let X;-1 = W and X; = W. If FIRST(flip,_,,left), then, within time 4, either 
X;-1 = Por X;= 5S. 


Proof. The four proofs start in the same way. Let s be a state of M satisfying the respective 
properties of items / or 2 or 3 or 4. Let A be an adversary of Unit-Time, and let a be an 
admissible timed execution of Qpreree(M,{s},A) Where the result of the first coin flip of process 
i—1, if it occurs, is left. 


1. By hypothesis and Lemma 6.3.4, 2 — 1 does not hold any resource at the beginning of a 
and has to obtain Res;_2 (its left resource) before pursuing Res;_1. From the definition 
of Unit-Time, i performs a transition within time 1 in a. If ¢— 1 does not hold Res;_; 
when 7 performs this transition, then 2 progresses into configuration S. If not, it must be 
the case that «— 1 succeeded in getting it in the meanwhile. But, in this case, since i — 1 
flips left, Res;_, was the second resource needed by 2— 1 and i— 1 therefore entered P. 


2. If X; = S within time 1, then we are done. Otherwise, process i— 1 performs a transition 
within time 1. Let a = a; ~ a such that the last transition of a, is the first transition 
taken by process i—1. Then Xj_1(fstate(a2)) = F and X;(fstate(az)) = W. Since process 
t— 1 did not flip any coin during a,, from the finite-history-insensitivity of Unit-Time 
and Item 7 we conclude. 


3. If X; = S within time 1, then we are done. Otherwise, process i— 1 performs a transition 
within time 1. Let a = a, ~ a such that the last transition of a, is the first transition 
taken by process 7 — 1. If X;_1(fstate(az)) = P then we are also done. Otherwise it must 
be the case that X;_1(fstate(a2)) = D and X;(fstate(a2)) = W. Since process 7 — 1 did 
not flip any coin during a1, from the finite-history-insensitivity of Unit-Time and Item 2 
we conclude. 
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4. If X; = S$ within time 1, then we are done. Otherwise, process i checks its left resource 
within time 1 and fails, process ¢ — 1 gets its right resource before, and hence reaches at 
least state S. Let @ = a1 ~ ag where the last transition of a1 is the first transition of a 
that leads process i — 1 to state 5. Then X;_1(fstate(a2)) = 5 and X;(fstate(a2)) = W. 
Since process i — 1 did not flip any coin during a1, from the finite-history-insensitivity of 
Unit-Time and Item 3 we conclude. | 


Lemma 10.6.5 Assume that X;-, € {Ep,R,T} and X; = W. If FIRST(£lip,_,, left), 
then, within time 4, either X;_1 = P or X; = S. 


Proof. Follows directly from Lemma 10.6.4 after observing that X;_, € {Ep, R,T} is equiva- 
lent to X;_-1 € {Er, R, FL W,S,D, P}. a 


The next lemma is a useful tool for the proofs of Lemmas 10.6.7, 10.6.8, and 10.6.9. It is just 
repeated from Section 6.3.4. 


Lemma 10.6.6 Let X; € {W, Ss} or X; € {Ep, R, F, D} with FIRST(flip,;,left). Further- 
more, let Xi41 € {W, S} or Xiqi € {Epn, RF, D} with FIRST(flip;,,,right). Then the 
first of the two processes i or i+ 1 testing its second resource enters P after having performed 
this test (if this time ever comes). 


Proof. By Lemma 6.3.4 Res; is free. Moreover, Res; is the second resource needed by both i 
and i+ 1. Whichever tests for it first gets it and enters P. | 


Lemma 10.6.7 /f X; = s and Xj41 € {W, Sh then, within time 1, one of the two processes 
tori+1 enters P. The same result holds if X; € {W, 5} and Xiz1 = 5. 


Proof. Being in state S, process i tests its second resource within time 1. An application of 
Lemma 10.6.6 finishes the proof. | 


Lemma 10.6.8 Let X; = S and Xi41 € {Ep, R, F, D}. If FIRST(flip,;,,,right), then, 
within time 1, one of the two processes 1 ori+1 enters P. The same result holds if X; € 


{Ep, R, FLD}, Xin = s and FIRST(£lip;, left). 


Proof. Being in state S, process i tests its second resource within time 1. An application of 
Lemma 10.6.6 finishes the proof. | 


Lemma 10.6.9 Assume that X;-1 € {Er,R,T}, Xi = W, and Xiz1 € {Er BR, FLW, D}. 
If FIRST(flip;_;,left) and FIRST(flip;,,,right), then, within time 5, one of the three 
processesi—1,t ort4+1 enters P. 


Proof. Let s be a state of M such that X;1(s) € {Er,R,T}, Xi(s) = W, and Xi41(s) € 
{Ep, R, F, W, D}. Let A be an adversary of Unit-Time, and let a be an admissible timed 
execution Of Qo rcree(M,{s},A) Where the result of the first coin flip of process t— 1 is left and 
the result of the first coin flip of process 7+ 1 is right. By Lemma 10.6.5, within time 4 either 
process 2 — 1 reaches configuration P in @ or process 7 reaches configuration s ina. Ifei-—1 
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reaches configuration P, then we are done. If not, then let a = ay ~ a2 such that Istate(a,) is 
the first state s! of a with X;(s’) = S'\. Ifz+1 enters P before the end of a1, then we are done. 
Otherwise, X;41(fstate(az)) is either in {W, Sh or it isin {Bp, R, F, D} and process 2+ 1 has 
not flipped any coin yet in a. From the finite-history-insensitivity of Unit-Time we can then 
apply Lemma 10.6.6: within time 1 process 7 tests its second resource and by Lemma 10.6.6 
process 2 enters P if process +1 did not check its second resource in the meantime. If process 
t+ 1 checks its second resource before process ¢ does the same, then by Lemma 10.6.6 process 
t+ 1 enters P. | 


Lemma 10.6.10 Assume that Xi42 € {Er, R,T}, Xiz1 = W, and X; € {Er Rk, FW, D}. 
If FIRST(flip;, left) and FIRST(flip;,.,right), then, within time 5, one of the three pro- 
cesses t,t +1 ori4+2, enters P. 


Proof. The proof is analogous to the one of Lemma 10.6.9. This lemma is the symmetric case 
of Lemma 10.6.9. | 


Proposition 10.6.11 Starting from a global configuration in G, then, with probability at least 
1/4, some process enters P within time 5. Equivalently: 


G—-P. 
1/4 
Proof. Lemmas 10.6.7 and 10.6.8 jointly treat the case where X; = Ss and Xj41 € {Ep, R, F, #} 
and the symmetric case where X; € {Fp, R, F, #} and X;4, = 5; Lemmas 10.6.9 and 10.6.10 
jointly treat the case where X; = W and Xj41 € {Er R,F, W,D} and the symmetric case 
where X; € {Ep, R,F, W, D} and Xj41 = Ww. 

Specifically, each lemma shows that a compound event of the kind FIRST (flip,,x) and 
FIRST(flip,,y) leads to P. Each of the basic events FIRST(flip;, x) has probability at least 
1/2. From Lemma 6.2.4 each of the compound events has probability at least 1/4. Thus the 
probability of reaching P within time 5 is at least 1/4. | 


We now turn to F ar GUP. The proof is divided in two parts and constitute the global 


argument of the proof of progress, i.e., the argument that focuses on the whole system rather 
than on a couple of processes. 


Lemma 10.6.12 Start with a state s of F. If there exists a process i for which X;(s) = F and 
(Xj-1, Niza) # (7, #), then, with probability at least 1/2 a state of GU P is reached within 
tame 1. 


Proof. If s € GUP, then the result is trivial. Let s be a state of F —(GUP) and let i be such 
that X;(s) = F and (Xj-1, Xi4i1) F (#, #). Assume without loss of generality that X;41 4 #, 
ie., Nig, € {Ep, R, F, #}. The case for X;1 4 # is similar. Furthermore, we can assume 
that Xi41 € {Er, R, F, D} since if X;41 € {W, S} then s is already in G. We show that the 
event schema FIRST ((flip;,left),(flip;,,,right)), which by Lemma 6.2.2 has probability 
at least 1/2, leads eventually to a state of GU P. Let A be an adversary of Unit-Time, and 
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let a be an admissible timed execution of Qyrevee(M,{s},A) Where if process ¢ flips before process 
i+ 1 then process 2 flips left, and if process 7+ 1 flips before process 7 then process 7 + | flips 
right. 

Then, within time 1, 7 performs one transition and reaches W. Let j € {i,i+ 1} be the 
first of 2 and 2+ 1 that reaches W and let s, be the state reached after the first time process 7 
reaches W. If some process reached P in the meantime, then we are done. Otherwise there are 
two cases to consider. If j7 = 7, then, flip; gives left and X;(s1) = W whereas Xj41 is (still) 
in {Fp, R, F, D}. Therefore, s; €G. If 7 =i+1, then flip,,, gives right and Xj41(s1) = Ww 
whereas X;(s1) is (still) F. Therefore, s1 € G. = 


Lemma 10.6.13 Start with a state s of F. If there exists a process i for which X;(s) = F and 
(Xj-1(s), Xiqa(s)) = (i, i). Then, with probability at least 1/2, a state of GU P is reached 
within time 2. 


Proof. The hypothesis can be summarized into the form (X;_-1(s), Xi(s), Xigi(s)) = (#, F, #). 
Since 2—1 and z+1 point in different directions, by moving to the right of ++1 there is a process 
k pointing to the left such that process & + 1 either points to the right or is in {Fp, R, F, P}, 
ie, Xp(s) € {W, S, D} and Xp41(s) € {ER R, FLW, SD, P}. 

If Xz(s) € {W, S} and Xp41(s) # P then s € G and we are done; if X,41(s) = P then 
s € P and we are done. Thus, we can restrict our attention to the case where X;(s) = Dd. 


We show that FIRST((flip,, left), (flip,,,,right)), which by Lemma 6.2.2 has proba- 
bility at least 1/2, leads to GUP within time 2. Let A be an adversary of Unit-Time, and let a 
be an admissible timed execution of Qorerec(M,{s},4) Where if process k flips before process k + 1 
then process k flips left, and if process k + 1 flips before process & then process k + 1 flips right. 

Within time 2 process k performs at least two transitions and hence goes to configuration 
W. Let j € {k,k + 1} be the first of k and k + 1 that reaches W and let s; be the state 
reached after the first time process 7 reaches W. If some process reached P in the meantime, 
then we are done. Otherwise, we distinguish two cases. If 7 = k, then, flip; gives left and 
Xx(81) = W whereas X;,41 is (still) in {Ep, R, F, #}. Thus, 1 €G. If7 =k+1, then flip,,, 
gives right and X;,41(s1) = W whereas X;(s1) is (still) in {D, F’}. Thus, s1 € G. = 


Proposition 10.6.14 Start with a state s of F. Then, with probability at least 1/2, a state of 
GUP is reached within time 2. Equivalently: 


F +GuP. 
1/2 
Proof. The hypothesis of Lemmas 10.6.12 and 10.6.13 form a partition of F. | 


Finally, we prove RT > FUGUP. 


Proposition 10.6.15 Starting from a state s of RT, then a state of F UGUP is reached 
within time 3 Equivalently: 


RT > FUGUP. 
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Proof. Let s be a state of RT. If s € FUGUP, then we are trivially done. Suppose that 
s¢ FUGUP. Then in s each process is in {Fp,R,W,S,D} and there exists at least process 
in {W,.S, D}. Let A be an adversary of Unit-Time, and let a be an admissible timed execution 
Of OQoreree(M,{s},A)- 

We first argue that within time 1 some process reaches a state of {5,D,F} in a. This 
is trivially true if in state s there is some process in {5,D}. If this is not the case, then all 
processes are either in Fp or R or W. Eventually, some process in R or W performs a transition. 
If the first process not in Fp performing a transition started in Fp or R, then it reaches F and 
we are done; if the first process performing a transition is in W, then it reaches S' since in s no 
resource is held. Once a process 7 is in {5 D,F'}, then within time 2 process i reaches either 
state F or P, and we are done. | 


10.7 Abstract Complexity Measures 


We have seen how to measure the expected time to satisfy a property. However, the technique 
can be extended to other kinds of measures of complexity. Specifically, let @ be a complexity 
measure on timed execution fragments that is additive under concatenation, i.e., 6(q1 ~ q2) = 
(m1) + &(q2). Then we can compute the expected ¢ rather than the expected time, where the 
@ of a state gq of A is defined to be o(qrgi ). We generalize the notation for timed progress 
statements by writing 

OC) ins vy! (10.34) 


ue 

P 
with the meaning that Pr dvs, (€u,4(c) ) > p, where the event schema EU! dlc) applied to a timed 
probabilistic execution fragment H returns the set of timed executions a of Qy where a state 
from U’ is reached within complexity c. More specifically, let Conesy: g¢c)(H ) be the set of 
minimal timed execution fragments gq of M such that CH is not empty, Istate(q) € U’, and 
b(qrg!) < c. Then, eur d(e\( 1) = Uge cones, ao(IDC4 Observe that time is just one of the 
possible complexity measures. . 

The same definition can be extended to sets of actions as we have done previously, and the 
concatenation theorem is still valid. 

The expected complexity of a finitely satisfiable event schema can be defined easily. Specifi- 
cally, if e is a finitely satisfiable event-schema and Cones( #) identifies the points of satisfaction 
of e, then for each probabilistic timed execution fragment H of M we define y4[e], the ex- 
pected complexity to satisfy e in HA, as follows. 


Epale] _ dog Cones(H) Pu(Cq\(O( gras )) if Pyle(H)| =1 (10.35) 


oe) otherwise. 
Then, a proposition similar to Proposition 10.5.1 can be proved. 


Proposition 10.7.1 Suppose that 


o(c) ! 
U ~Adus U (10.36) 
U => (U Unless U'), 
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Figure 10-1: An example of the use of €. 


and suppose that Advs is finite history insensitive and that 36 ¢ Q4(s) for each A € Advs and 
each s EU. Then, 


Ev, Advs,gle] < ¢ + pEus advs,ale] + 1 — p)(E + Ev,Ads,elel), (10.37) 


where 


g = SUP ge t-frag*(M)|Istate(q)eU (supgsg (inf ging cat<q($(a"P9)))) . (10.38) 


Proof. This proof has the same structure as the proof of Proposition 10.5.1. Here we describe 
in detail only the main differences. In particular, we show part of the derivation from Equa- 
tion (10.16) to Equation (10.21), where the constant € is used. Observe that if we use ¢ to 
express time complexity, then € = 0. 

From (10.35) the expected complexity for success for e is 


Exglel= >> Pu[C,]6(q>a6"). (10.39) 
q€ Cones(H) 


For each d > 0, let Conesg be a function that expresses the event of reaching complexity d as 
a union of disjoint cones. From the definition of a probabilistic timed execution, we know that 
Conesq exists and, from (10.38), we know that for each probabilistic timed execution fragment 
H and each q € Conesy(H), d < o(qrqi!) < d+ €. Let € be any positive number. Following 
the same derivation as in the proof of Proposition 10.5.1, we obtain 


qeO1 qEO2 


Enale] < (e+ € (5 Pa Py[Cq|Eu'ads,ole p) 4 (= Pu Pr[Cq](€ + Ev, advs,ale | . (10.40) 


a 
One of the novel aspects of Proposition 10.7.1 is the constant €. Roughly speaking, € gives us a 
lower bound to the minimum complexity increase that we can obtain by moving along a timed 
execution fragment. 


Example 10.7.1 (Why € is necessary) For example, if the abstract complexity that we use 
is the number of discrete actions that appear in a timed execution fragment, then € = 1. In fact, 
whenever we perform a discrete action, the complexity increases by 1. Figure 10-1 shows an 
example where € = 1 and where Equation (10.37) is invalidated if we do not include €. Denote 
the probabilistic timed execution fragment of Figure 10-1 by H. Let U be {so}, U’ be {s:}, and 
let e express the property of reaching U’. Let Advs contain only one adversary that generates H 
when applied to s9. Let ¢ count the number of external actions in a timed execution fragment 


¢(1) 


(no time-passage actions in H). Then, it is immediate to verify that the statement U ar: U' is 
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valid in H and that also U > (U Unless U’) is valid. By applying Equation (10.37) with € = 1, 
we obtain 


Ev, Adus,olé] < t+ 1/20. + Ev sas,olel), (10.41) 


which leads to Bu, advs,s[e] < 3. If we did not use € in Equation (10.37) we would have obtained 
Ev, Adus,ol€] < 2. We now show that F7,[e] = 3. In fact, 


1 1 1 1 


By rearranging the terms, we obtain 
1/1 2 2. 2 
E _ 2f(i4524 24 244...). 10.43 
ned =x (5+Gtetiet (10.43) 


Recall that 3>;,) 1/2' = 2. Thus, by rearranging the terms again, 


Engl = 2+1/2(5 : 4+ ) 33. (10.44) 
Roughly speaking, the transition relation of H is structured in such a way that whenever the 
experiment of reaching U’ from U fails, the system looses one additional complexity unit during 
the random draw. In the proof of Proposition 10.7.1 this phenomenon is detected when we define 
the partition 0, and O02. To make sure that 0, and ©» partition an event with probability 1 
and that ©, captures all the places where U' is reached within time t, @2 must be based on 
states reached after time t. In the probabilistic execution H of this example the states of O» 
have complexity ¢+ 1. | 


10.8 Example: Randomized Agreement with Time 


Using abstract complexity measures it is possible to show that the randomized agreement 
algorithm of Ben-Or guarantees agreement within an expected exponential time. This is not 
an exceptional complexity result, but it corresponds to the time complexity of the algorithm. 

In more detail, we add time to the probabilistic automaton that describes Ben-Or’s protocol 
in the same way as we have done for the Dining Philosophers algorithm of Lehmann and Rabin. 
In this case each adversary is required to schedule every process that enables some transition 
within time 1 from every point. Then we show an upper bound linear in st on the time it 
takes to all processes to complete a specific stage st. Finally, we derive an upper bound on 
the expected number of stages it takes for all processes to decide. This is achieved by defining 
an abstract complexity on the timed executions of M that checks the highest stage reached at 
every point. A direct extension of the untimed proof without abstract complexities would not be 
possible. In fact, given a reachable state s, the validity of the progress statement of Chapter 6 
relies on completing the highest stage reached in s, and we cannot establish any useful upper 
bound on the time to complete such stage: there is no useful bound on the difference between 
the highest and the lowest stages reached in s, and the adversary may stop the processes with 
the highest values of st. We start by proving the upper bound on the time it takes to each 
process to complete some stage st. 
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Lemma 10.8.1 There is a constant d such that, for each stage st, each process completes stage 
st within time d- st. 


Proof. Let d, be the maximum time it takes to each process from the moment it reaches a new 
stage st to the moment it broadcasts its value and its value is delivered; let dz be the maximum 
time it takes to each process to broadcast and deliver its second message after receiving enough 
messages from the first round; let d3 be the maximum time it takes to each process to move to a 
new stage once it has received enough messages from the second round. Then d = d; + dz +ds3. 
Since we have not defined formally M, we cannot say explicitly what is the value of d. 

We show the result by induction on st where for the base case we assume that st = 0 
and that stage 0 is completed by time 0. By induction, by time d- st each non-faulty process 
has completed round st. Then, by time d, + d- st each non-faulty process has broadcasted 
and delivered its first round message, and thus every non-faulty process has received enough 
messages for the first round of stage st +1. Within additional time dz each non-faulty process 
delivers its second message, and within additional time d3 each non-faulty process reaches stage 
st + 2, i.e., within time d(st + 1) each non-faulty process completes stage st + 1. | 


For each finite timed execution fragment a of M define ¢(a), the stage complexity of a, to 
be maz-stage(Istate(a)) — max-stage(fstate(a)), where for each state s, maz-stage(s) is the 
maximum stage that is reached in s by some process. Observe that this complexity measure is 
an upper bound to the stage at which some process decides since if at state s the first process 
has just decided, then max-stage(s) is not smaller than the stage of the process that has decided. 
Thus, an upper bound on the expected ¢ for the decision of the first process is an upper bound 
on the expected stage at which the first process decides. We show the following two statements. 


BO tin FUO. (10.45) 


$2) «@ (10.46) 
1/2” 
Then, by combining (10.45) and (10.46) with Theorem 5.5.2, we obtain 


Bo, (10.47) 


—+ 
1/ar 


F 


From Proposition 10.7.1, we obtain 


Eg Unit-Time,oleo] < 3+ (1 -— 1/2")(1+ Eg, unit-Time,leo]); (10.48) 
where 1 is the value of € given by (10.38). By solving Equation (10.48) we obtain 
Ep, Unit-Time,sleo] < 2"** — 1. (10.49) 


Since if a process decides at stage st then each other non-faulty process decides within stage 
st +1, then we can derive that the expected stage by which every process decides is at most 
2"+?, and thus, from Lemma 10.8.1, each process decides within expected time d-2”*!. 

The proofs for (10.45) and (10.46) have the same structure as the corresponding proofs 
for the untimed case. Recall that the proof of (10.45) consider the maximum stage st of a 
reachable state s and states that eventually stage st + 1 is reached, at which time a state of F 
is reached. The proof of (10.46) states that a specific coin lemma leads a process to decide by 
stage maz-stage(s)+1. Then, since if a process decides a stage st each process decides by stage 
st + 1, the complexity of the state where the first process decides is at most maz-stage(s) + 2. 
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10.9 Discussion 


To our knowledge this is the first time that statements similar to our timed progress statements 
have been used for the analysis of the performance of a randomized distributed algorithm. In 
particular, we have been able to prove similar results only because we have studied techniques to 
prove properties that hold with some probability different than 1. This should be a sufficiently 
strong reason to pursue additional research on methodologies (automatic or not) for the analysis 
of properties that hold with probabilities different than 1. The work of Hansson [Han94] and 
the algorithm that Courcoubetis and Yannakakis present in [CY90] are in this direction. 
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Chapter 11 


Hierarchical Verication Timed 
Trace Distributions 


11.1 Introduction 


In this chapter we extend the trace distribution preorder of Chapter 7 to the timed framework. 
The main difference is that we use timed traces rather than traces. A timed trace contains the 
sequence of discrete actions that occur within a timed execution plus the time of occurrence 
of each action and the time at which the observation ends. That is, in a timed execution we 
observe at what time each external action occurs and, if finitely many actions occur, how much 
time elapses after the occurrence of the last action. 

We define a preorder relation based on timed trace distribution inclusion, and we characterize 
the coarsest precongruence that is contained in the timed trace distribution preorder by using 
a timed principal context, which is just the principal context of Chapter 7 augmented with 
arbitrary time-passage self-loop transitions from its unique state. Most of the proofs follow 
directly from the results already proved in Chapter 7, since in several cases it is sufficient to 
study ordinary trace distributions in order to derive properties of timed trace distributions. 


11.2 Timed Traces 


We start by defining the main object of observation, i.e., timed traces. The definition of a timed 
trace that we give in this section is taken directly from [LV95]. 


Timed Sequence Pairs 


Let K be any set that does not intersect Rt. Then a timed sequence over K is defined to be a 
(finite or infinite) sequence y over K x R2° in which the time components are nondecreasing, 
ie., if (k,t) and (k’,t’) are consecutive elements in y then t < t’. We say that y is Zeno if it is 
infinite and the limit of the time components is finite. 

A timed sequence pair over K is a pair 3 = (7,t), where y is a timed sequence over A’ and 
t € R2° U {oo}, such that ¢ is greater than or equal to all time components in y. We write 
seq(3), and Itime(3) for the two respective components of 3. We denote by tsp( i) the set of 
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timed sequence pairs over kK. We say that a timed sequence pair / is finite if both seq(@) and 
ltime(3) are finite, and admissible if seq(3) is not Zeno and Itime(3) = ov. 

Let 3 and 2’ be timed sequence pairs over A with @ finite. Then define 3; 3’ to be the timed 
sequence pair (seq(J)7, Itime(3) + ltime(2")), where 7 is the modification of seq(’) obtained 
by adding /time(/3) to all the time components. If 3 and ’ are timed sequence pairs over a set 
K, then ( is a prefix of 3’, denoted by @ < 6’, if either @ = G’, or @ is finite and there exists a 
timed sequence pair 3” such that 3’ = 6; 8". 


Lemma 11.2.1 < ts a partial ordering on the set of tamed sequence pairs over K. | 


Now we describe how to translate from a sequence over K UT, and ordinary trace, to a timed 
sequence pair over K. First, if @ is any sequence over AK U RT, then we define the time of 
occurrence of any K-element in @ to be the sum of all the reals that precede that element in 
GB. We also define ltime(3) to be the sum of all the reals in 3. Finally, we define t-trace(3) to 
be the timed sequence pair (7, /time(3)), where 7 is the subsequence of { consisting of all the 
elements of , each paired with its time of occurrence. 

If @ is a sequence over K UR* then we say that 3 is admissible if the sum of the positive 
reals in @ is infinite. 


Lemma 11.2.2 If § is a finite or admissible timed sequence pair then t-trace(trace(3)) = 3. 


Lemma 11.2.3 If 3 is a sequence over K UT then 8 is admissible if and only if t-trace( 3) 
is admissible. | 


Timed Traces of Timed Probabilistic Automata 


Suppose that @ = wodjw1dqw2--- is a timed execution fragment of a timed probabilistic au- 
tomaton M. For each a;, define the time of occurrence t; to be 7, <; Itime(w;), ie., the sum of 
the lengths of all the trajectory intervals preceding a; in a. Let y be the sequence consisting of 
the actions in @ paired with their times of occurrence: 

7 = (a1, t1)(G2, ta) ++. 
Then t-trace(a), the timed trace of a, is defined to be the pair 

(y | (vis(M) x RT), Itime(a)). 


Thus, ¢-trace(a) records the occurrences of visible actions together with their times of oc- 
currence, and together with the time spanned by a. Note that neither internal actions nor 
time-passage actions appear explicitly in the timed trace of a. 


Proposition 11.2.4 If a is a timed execution fragment of M then t-trace(a) is a timed se- 
quence pair over vis(M). = 


Proposition 11.2.5 Let a be a timed execution fragment of M, and let trace(a) denote the 
ordered sequence of external actions that appear in a. Then, t-trace(a) = t-trace(trace(a)). ™ 


Proposition 11.2.6 If a = a; ~ ag is a timed execution fragment of M, then t-trace(a) = 
t-trace(ay ); t-trace(az). = 
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We write t-traces(M) for the set of all timed traces of M, t-traces*(M) for the set of finite 
timed traces of M, and t-traces*(M) for the set of admissible timed traces of M, 

The timed traces of a probabilistic timed automaton M can be characterized also in terms 
of its time-enriched executions or in terms of its ordinary executions. Specifically, if a is a time- 
enriched execution of M, then let t-trace(a) denote t-trace(t-exec(a)), and if a is an execution 
of M, then let t-trace(a) denote t-trace(trace(a)). The following proposition holds. 


Proposition 11.2.7 Let M be a probabilistic timed automaton. 


1. Ifa is a time-enriched execution of M, then there is a timed execution a’ of M such that 
t-trace(a) = t-trace(a’). 


2. Ifa is a timed execution of M, then there is a time-enriched execution a! of M such that 
t-trace(a) = t-trace(a’). 


3. Ifa is a timed execution of M, then there is an execution a’ of M such that t-trace(a) = 
t-trace(a’), 


4. If a is an execution of M, then there is a timed execution a! of M such that t-trace(a) = 
t-trace(a’), 


Proof. 


1. Let a’ be t-exec(a). Then, t-trace(a) = t-trace(a’) by definition. 


2. Let a@ be woaywyidg---. If a is a finite timed execution or an infinite sequence, then let 
a’ = fstate(wo) ~ ay ~ a2 ~--++, where for each 4, 
ee ee ajfstate(w;) if w;-1 has domain [0, 0], 
‘| fstate(w;_1 )itime(w;_1 )w;_1a;fstate(w;) otherwise; 


if @ = wodywyag++-d,w, and the domain of w, is right-open, then let a’ = fstate(wo) ~ 
a, ~+++7 Qn ~ a} 41, where the a;’s are defined above and af... = wodiw}dgw)--- is an 
infinite sequence such that wowiw,--- = w,. It is immediate to verify that a and a’ have 
the same timed trace since a = t-erec(a’). 


3. Let a@ be woayw iag---. If a is a finite timed execution or an infinite sequence, then let 
a’ = fstate(wo) ~ a1 ~ a2 ~---+, where for each 3, 
aK Istate(wj_y )a;fstate(w;) if w;_; has domain [0, 0], 
‘| fstate(w;_1 )ltime(w;_1 )lstate(w;_1 )a;fstate(w;) otherwise; 


if @ = woaywyd2-+-ad,W, and the domain of w, is right-open, then let a” = fstate(wo)*a1~ 
“+a, ~al,44, where the a;’s are defined above and af,,, = fstate(Wn )dywn (dy down (di + 
dz)-+- is an infinite sequence such that 5°; d; = Itime(w,). It is immediate to verify that 
a and a’ have the same timed trace. 
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4, Given a = s9a15142---, build a time-enriched execution a” by replacing each state s; with 
a trajectory for (s;-1,4@;,5;) whenever a; is a time-passage action. Then, t-trace(a) = 
t-trace(a’’). Item 2 is enough to conclude. = 


The bottom line of the proposition above is that for the study of the timed traces of a probabilis- 
tic timed automaton it is not necessary to observe the trajectories spanned by a computation. 
The points of occurrence of discrete actions are sufficient. 


11.3. Timed Trace Distributions 


In this section we define the timed trace distributions of a probabilistic timed automaton and we 
extend the action restriction operation. The main result is that it is possible to study the timed 
trace distributions of a probabilistic timed automaton M by considering either its probabilistic 
executions, or its probabilistic time-enriched executions, or its probabilistic timed executions. 


11.3.1 Three ways to Define Timed Trace Distributions 


We now define the timed trace distribution of a probabilistic execution, of a probabilistic time- 
enriched execution, and of a probabilistic timed execution of a probabilistic timed automaton. 
The definitions are given in the same style as for the untimed case. Furthermore, we show that 
the three definitions lead to the same collection of timed trace distributions. This enforces the 
remark that for the study of the timed trace distributions of a probabilistic timed automaton 
it is not necessary to observe the trajectories spanned by a computation. 


Timed Trace Distribution of a Probabilistic Execution 


Let H bea probabilistic execution of a probabilistic timed automaton M, and let f be a function 
from Qy to Q = tsp(vis( M)) that assigns to each extended execution its timed trace. The timed 
trace distribution of H, denoted by t-tdistr(H), is the probability space completion((Q, F, P)) 
where F is the o-field generated by the cones C'g, where (3 is a finite timed sequence pair of 
tsp(vis(M)), and P = f(Py). Note that from Proposition 3.1.4 f is a measurable function 
from (Qy, Fy) to (Q,F). 


Timed Trace Distribution of a Probabilistic Time-Enriched Execution 


Let H be a probabilistic time-enriched execution of a probabilistic timed automaton M, and 
let f be a function from Qy to Q = tsp(vis(M)) that assigns to each time-enriched extended 
execution its timed trace. The timed trace distribution of H, denoted by t-tdistr(H), is the 
probability space (Q, 7, P) where F is the o-field generated by the cones Cg, where / is a finite 
timed timed sequence pair of tsp(vis(M)), and P = f(Py). Note that from Proposition 3.1.4 
f is a measurable function from (Qy, Fy) to (Q,F). 


Timed Trace Distribution of a Probabilistic Timed Execution 


Let H be a probabilistic timed execution of a probabilistic timed automaton M, and let f 
be a function from Qy to Q = tsp(vis(M)) that assigns to each timed extended execution 
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its timed trace. The timed trace distribution of H, denoted by t-tdistr(H), is the probability 
space (Q, 7, P) where F is the o-field generated by the cones Cg, where is a finite timed 
timed sequence pair of tsp(vis(M)), and P = f( Py). Note that from Proposition 3.1.4 f is a 
measurable function from (Qy, Fy) to (Q, F). 


Equivalence of the Definitions 


We now show that the three definitions of a timed trace distribution lead to the same collection 
of timed trace distributions when applied to a probabilistic timed automaton (cf. Proposi- 
tions 11.3.2 and 11.3.4). Thus, we can freely denote a generic timed trace distribution by D 
and denote the timed trace distributions of a probabilistic tomed automaton M by t-tdistrs(M). 


Lemma 11.3.1 Let H be a probabilistic time-enriched execution of a probabilistic timed au- 
tomaton M. Then, t-tdistr( H) = t-tdistr( sample(H)). 


Proof. Let D be t-tdistr(H) and let D’ be t-tdistr(sample(H)) Consider a finite timed trace 
G. From the definition of t-tdistr(), 


Pp[Cg] = Psampte(H)@ € Ysample(H) | F < t-trace(a)}}. (11.1) 


Since Cg is a finitely satisfiable event, there is a set of © of states of sample(H) such that for 
each element g of O, 6 < t-trace(q), and such that 


{a € Qyampie(Hy |B < ttrace(a)} = UgeoO er), (11.2) 
Thus, 
Pp: [Cs] = SOP. sample(H ogemele)), (11.3) 
qEo 


From Equation (9.55), Equation (11.3) becomes 


Ppi[C] = So PalC#). (11.4) 


qg€ésample—1(O) 
Observe that sample'(@) is a characterization of C'g for D, and thus, 
Pp [Cg] = Po[Ce]. (11.5) 
This completes the proof. | 


Proposition 11.3.2 Let M be a probabilistic timed automaton. Then, for each probabilis- 
tic time-enriched execution H of M there exists a probabilistic execution H' of M such that 
t-tdistr(H) = t-tdistr(H'), and for each probabilistic execution H of M there exists a proba- 
bilistic time-enriched execution H' of M such that t-tdistr(H) = t-tdistr(H'). 


Proof. Follows directly from Propositions 9.3.6 and 9.3.7, and from Lemma 11.3.1. | 


Lemma 11.3.3 Let H be a probabilistic time-enriched execution of a probabilistic timed au- 
tomaton M. Then, t-tdistr(H) = t-tdistr(t-sample(H)). 
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Proof. Let D be t-tdistr(H), and let D’ be t-tdistr(t-sample(H)). Consider a finite timed 
sequence pair D of tsp(vis(M)). From the definition of t-tdistr, 


Pp[Cg] = Pul{a € Qy | 8 < t-trace(a)}]. (11.6) 


From the definition of t-erec(Py), 


Pp[Cg] = Precxec(Py)IL € 4. evee(H) | p < t-trace(a)}]. (11.7) 
With a similar analysis, 

Pp Cg] = Pt.sample(H) Ae € (2+. sample(H) | p < t-trace(a)}]. (11.8) 
Since from Proposition 9.3.11 t-exec(Py) = P.sample(H), and since the events of (11.7) and (11.8) 
are unions of countably many disjoint cones, we conclude that Pp[C'g] = Pp:|[C4]. = 


Proposition 11.3.4 Let M be a probabilistic timed automaton. Then, for each probabilistic 
time-enriched execution H of M there exists a probabilistic timed execution H' of M such that 
t-tdistr(H) = t-tdistr(H'), and for each probabilistic timed execution H of M there exists a 
probabilistic time-enriched execution H' of M such that t-tdistr( H) = t-tdistr(H’). 


Proof. Follows directly from Propositions 9.3.8 and 9.3.9, and from Lemma 11.3.3. | 


Proposition 11.3.5 Let H, and Hy» be two equivalent probabilistic time-enriched executions of 
a probabilistic timed automaton M. Then, t-tdistr( H,) = t-tdistr( H2). 


Proof. From Proposition 9.3.10, t-sample(H,) = t-sample(H2), and from Lemma 11.3.3, 
tdistr(H,) = tdistr(t-sample(H,)) and tdistr(H2) = tdistr(t-sample(H2)). Thus, combining 
the observations above, t-tdistr(H,) = t-tdistr(H2). = 


11.3.2. Timed Trace Distribution of a Trace Distribution 


Given a trace distribution of a probabilistic timed automaton, it is possible to define its timed 
trace distribution as we have done for ordinary traces. Thus, let D be a trace distribution of a 
probabilistic automaton, and let f be a function from Qp to N = {t-trace(B) | B € Qp} that 
assigns to each trace its timed trace. The timed trace distribution of D, denoted by t-tdistr(D), 
is the probability space completion((Q,F,P)) where F is the o-field generated by the cones 
Cy, where f is a finite timed trace, and P = f(Pp). Note that from Proposition 3.1.4 f is a 
measurable function from (Qp, Fp) to (Q, F). 


Proposition 11.3.6 Let H be a probabilistic execution of a timed probabilistic automaton M. 
Then, t-tdistr(H ) = t-tdistr(tdistr(H)). 


Proof. Let D be t-tdistr(H), and let D’ be t-tdistr(tdistr(H)). We show first that D and D’ 
have the same sample space. Then, we show that they assign the same probability to each cone. 

To show that D and D’ have the same sample space, it is enough to show that for each 
timed sequence pair § of tsp(vis(M)) thehre is a trace 9’ of ext(M)* U ext(M)” such that 
t-trace(3’) = B. Let (3 = (a1, t1)(a2, tz), (a3,t3)---,t). If seq(Z) is an infinite sequence, then 
let B' = B1203---, where for each 7, if tj4, = t;, then 3; = a;, and if tj4, > t;, then 6; = 
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aj(ti41 —t;). If seq(9) is a finite sequence, i.e., seq( 9) = (a1, t1) (da, tz), (a3, t3) +++, (Gn, ty) then 
B' = 31 G283---Bn—1}, where the (,’s are defined above, and //, is a, if th = t, a,(t — t,) if 
0<t—t, < o, and a, followed by the infinite sequence of 1’s if f = oo. It is easy to verify 
that in every case t-trace(3’) = 3. 

To show that D and D’ assign the same probability to each cone, let 3 be a finite timed 
trace. From the definition of ¢-tdistr and tdistr, 


Pp [Ce] = Pal{a € Qy | 8 < t-trace(trace(a))}). (11.9) 
From Proposition 11.2.5, (11.9) becomes 

Pp [Cg] = Prl{a € Qy | 6 < t-trace(a)}], (11.10) 
which is the definition of Pp[Cg]. = 


11.3.3. Action Restriction 


Finally, we extend the action restriction operator to timed trace distributions. Let M be a 
probabilistic timed automaton, and let V be a set of visible actions of M. For each timed trace 
OB = (7,t) of M, let @ | V be the pair (7’,t) where 7’ is obtained from 7 by removing all the 
pairs whose action is in V. Let D be a timed trace distribution of /. Define D [ V to be the 
timed trace distribution (2,7, P) where Q = Qp | V, F is the o-field generated by the cones 
C'g, where (7 is a finite timed trace, and P = Pp [| V. Note that from Proposition 3.1.4 [ V is a 
measurable function from (Qp, Fp) to (Q, F). Action restriction commutes with the operation 
of taking a timed trace distribution of a trace distribution. 


Proposition 11.3.7 Let D be a trace distribution of a probabilistic timed automaton M, and 
let V be a set of visible actions of M. Then, t-tdistr(D | V) = t-tdistr(D) [ V. 


Proof. Let D’ be t-tdistr(D [| V), and let D” be t-tdistr(D) [| V. Let @ be a finite timed trace. 
By applying the definitions of t-tdistr and of |, we obtain the following two equations. 


Pp [Ca] = Ppo[{8" € Qn | B < t-trace(G’ | V)}]. (11.11) 
Pp»[Cg] = Po[{" € Qn | B < t-trace(B") | V}¥]. (11.12) 
Observe that for each 9’ of Qp, t-trace(f’ | V) = t-trace(B") | V. Thus, the right expressions 
of (11.11) and (11.12) denote the same value. That is, Pp) [Cg] = Pp» [Cg]. = 


11.4 Timed Trace Distribution Precongruence 


Let 1,1, Mz be two probabilistic timed automata with the same external actions. The timed 
trace distribution preorder is defined as follows. 


My, Cry Mo iff t-tdistrs(M)) C t-tdistrs( M2). 


As for the untimed case, the timed trace distribution preorder is not a precongruence. A 
counterexample can be created directly from the counterexample of Chapter 7 by augmenting 
the probabilistic automata of Figure 7-4 with arbitrary self-loop time-passage transitions from 
their deadlock states (the states that do not enable any transition). Thus, we define the 


timed trace distribution precongruence, denoted by Epc;, as the coarsest precongruence that is 
contained in the timed trace distribution preorder. 
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11.5 Alternative Characterizations 


The timed trace distribution precongruence can be characterized by a timed version of the 
principal context of Chapter 7. Namely, let the timed principal context, denoted by Cp be 
the principal context of Figure 7-6 augmented with self-loop time-passage transitions for each 
time-passage action d. Then, the following holds. 


Theorem 11.5.1 My Cpc: Me iff Mi||Cp Cpt M2||Cep. 


Thus, if we define the principal timed trace distributions of a probabilistic timed automaton 
M, denoted by pt-tdistrs(.M), to be the timed trace distributions of M||Cp, then we get the 
following. 


Corollary 11.5.2 My, Cpc; Mo iff ext( Mi) = ext( M2) and pt-tdistrs( M1) C pt-tdistrs( M2). 
| 


The rest of this section is dedicated to the proof of Theorem 11.5.1. The structure of the proof 
follows the same lines as the proof of Theorem 7.5.1, where only one additional transformation 
step is added: a distinguishing context is transformed into a new time-deterministic context 
where each state enables either discrete actions only or time-passage actions only. A time- 
deterministic context is a probabilistic automaton such that for each state s and each time- 


passage action d, if s 4, 8, and s 4, 82, then sy = 5s. All the lemmas except for one are 
proved by reducing the problem to the untimed framework. 


Lemma 11.5.3 Let C be a distinguishing context for two probabilistic timed automata My, and 
M2. Then there exists a distinguishing context C' for My and Mz with no discrete actions in 
common with My and M,. C" is called a separated context. 


Proof. The context C’ is built from C in the same way as in the proof of Lemma 7.5.3. The con- 
structions clp and exch work as well (they never exchange transitions involving time-passage), 
and the proof is carried out at the level of probabilistic executions rather than probabilistic 
timed executions. 

Specifically, let D be a timed trace distribution of M||C that is not a timed trace distri- 
bution of M2||C. Consider a probabilistic execution Hy of M,||C such that t-tdistr(H,) = D, 
and consider the scheduler that leads to H;. Apply to M,||C’ the same scheduler with the 
following modification: whenever a transition ((s1,¢),a,P1 @ P) is scheduled in M,||C, sched- 
ule ((s1,¢), a1, P((s1,¢’))), where c’ is ¢(..4,p), followed by ((s1,¢'),@,P1 ® D(c’)), and, for each 
si € 1, followed by ((s},¢’), a2, D(s,)®@P). Denote the resulting probabilistic execution by Hj 
and the resulting timed trace distribution by D’. From Lemma 7.5.3, tdistr(H,) = tdistr(H) { 
vis(M,||C), and thus, from Propositions 11.3.6 and 11.3.7, D = D' [ vis(M,||C). 

Suppose by contradiction that it is possible to obtain D’ from M2||C’. Consider the scheduler 
that leads to D’ in M3||C’, and let H4 be the corresponding probabilistic execution. Then, from 
Lemma 7.5.3, clp(exch(H3)) is a probabilistic execution of M3||C’, and tdistr(clp(exch(H$))) = 
tdistr(H5) } acts(M,||C). From Propositions 11.3.6 and 11.3.7, D = t-tdistr(clp(exch(H4))), 
which is a contradiction. | 


Lemma 11.5.4 Let C be a distinguishing separated context for two probabilistic timed automata 
M, and Mz. Then there exists a distinguishing cycle-free separated context C’ for My, and M2. 
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Proof. The context C’ can be built by unfolding C. Every scheduler for C' can be transformed 
into a scheduler for C’ and vice versa, leading to the same timed trace distributions. | 


Lemma 11.5.5 Let C be a distinguishing cycle-free, separated context for two probabilistic 
timed automata M, and Mg. Then there exists a distinguishing time-deterministic, cycle-free 
separated context C’ for My, and My that from any state enables either time-passage actions 
only or discrete actions only. 


Proof. The context C” is built from C as follows: 


1. for each time-passage transition s —1. sof C and each trajectory w for s , s’, add an 
action start,, and an action end,; 


2. for each time-passage transition s —1. s! of C and each trajectory w for s ~, s', adda 


: “ys start “ys end 
collection of new states {s,;|0<t< d}, atransition s —— s,,9,a transition s,q¢ —* s’, 
? — — ? ? 


and for each 0 << t < t <d,a transition s,, 4 it Su tl} 
3. remove all the time-passage transitions leaving from states of C. 


Let D be a timed trace distribution of M;||C that is not a timed trace distribution of M||C. 
Consider a probabilistic execution Hy, of M,||C such that t-tdistr(H,) = D, and consider the 
scheduler that leads to H,. Apply to M,||C’ the same scheduler with the following modification: 


whenever a time-passage transition s 1. 6! is scheduled, choose a trajectory w for s 1, gl 
and schedule start,,, followed by d, and followed by end,,. Denote the resulting probabilistic 
execution by Hj and the resulting timed trace distribution by D’. Then, 


D' | acts(M,||C) = D. (11.13) 


To prove (11.13) we prove first that tdistr(H{) | acts(.M,||C) = tdistr(H,), and then we apply 
Propositions 11.3.6 and 11.3.7. To prove that tdistr( Hj) [ acts(.M,||C) = tdistr(H,) we define 
a construction tclp to be applied to probabilistic executions of M;||C’ where each occurrence of 
a start action is followed eventually by the corresponding end action with probability 1. 

Let H' be a probabilistic execution of M;||C’ where each occurrence of a start action is 
followed eventually by the corresponding end action with probability 1, and denote tclp( H’) by 
H. For each state ¢ of H’, let tclp(q) be obtained from gq by replacing each state of the form s., 4 
with the state w(t), by removing each occurrence of a start action together with its following 
state, and by removing each end action together with its following state. Then, 


states(H) = telp(states(H')). (11.14) 


Let (¢, P) be a restricted transition of H’, and suppose that no start or end action occurs. Let 
Q! = {(a, telp(q’)) | (a,q’) € Q3, and for each (a,q”) € Q', let P'[(a,q")] = Pla x telp~'(q’)], 
where telp—'(q) is the set of states q’ of H’ such that telp(q’) = q. Then the transition 
tclp((q,P)) is defined to be 


telp((q,P)) = (telp(q),P). (11.15) 
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For the transition relation of H, consider a state q of H, and let min(tclp~'(q)) be the set of 
minimal states of telp~'(q) under prefix ordering. For each state q € telp—'(q), let 


—telp—*(q) 4 Pu [Ca 


P i a 7s (11.16) 
, yo g'emin(telp—2(q)) PHC] 
The transition enabled from gq in A is 
Sy pt?) PH Tacts(Mil|C)}telp(trt” } acts( Mil|C)). (11.17) 


q’€telp—1(q) 


The probabilistic execution H satisfies the following properties. 


a. H is a probabilistic execution of M;,||C. 


The fact that each state of H is reachable can be shown by a simple inductive argument; 
the fact that each state of H is a finite execution fragment of M;||C follows from a simple 
analysis of the definition of telp. 

From (11.17) it is enough to check that for each state q’ of H’, the transition telp( tr t 
acts(M,;||C)) is generated a combined transition of M;||C. Since tr is a transition of 


H', (tr | acts(.M;||C)) can be expressed as q/~ tr, where tr is a combined transition of 
M,||C’ and no start or end action occurs in tr. Let tr’ be obtained by substituting each 
state of the form s,, with w(t) in tr. Then, tr’ is a combined transition of M||C, and, 
from the definition of tclp, telp( tr t acts(.M;||C)) = telp(q’) > tr’. 


. For each state q of H, 


Pu[Cy] = S- Pu [Cy]. (11.18) 


q’Emin(telp—(q)) 


This is shown by induction on the length of g. If q consists of a start state only, then 
the result is trivial. Otherwise, from the definition of the probability of a cone, Equa- 
tion (11.17), and a simple algebraic simplification, 


Pi[Coas] = PulCy] ( So ph O PHla x a) , (11.19) 


q'€telp—+ (q) 


Observe that for each q’ € telp~'(q) the set Qi N ({a} x telp~!(qas)) contains only one 
element, say (a, gas”), and thus Pry [Cy PP [ax tclp~'(qas)] gives Py [Cq'as]. Moreover, 
observe that the states of min(telp~'(qas)) are the states of the form described in Equa- 
tion (11.19) (simple cases analysis). Thus, by applying induction to (11.19), using (11.16), 
simplifying algebraically, and using the observations above, 


Pu[Cqas] = S- Pu [Cy]. (11.20) 


q’€min(telp—"(qas)) 
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c. tdistr( H) = tdistr( H') | acts(M;\|C). 


Let 6 be a finite trace of H or H’. Then {a € Qy: | 8 < trace(a) f acts(M;||C)} can be 
expressed as a union of disjoint cones UzeoC, where 


0 = {¢ € states( H’) | trace(q) | acts(M;||C) = 8, lact(q) = lact(3)}. (11.21) 
The set tclp(O) is the set 
tclp(O) = {q¢ € states( H) | trace(q) = G, lact(q) = lact(3)}, (11.22) 


which is a characterization of {a € Qy | 8 < trace(a)} as a union of disjoint cones. Ob- 
serve that min(telp—'(telp(®))) = @. Moreover, for each q # qo of telp(®), telp~'(q1) N 
telp~(q2) = 0. Thus, from (11.18), Pry[UgeoCg] = Pr[Ugeteip(e)Cq]- This is enough to 
conclude. 


To complete the proof of (11.13) it is enough to observe that Hy = tclp( Hj). Property (11.13) 
is then expressed by property (c). 

Suppose by contradiction that it is possible to obtain D’ from M2||C’. Consider the scheduler 
that leads to D’ in M3||C’, and let H4 be the corresponding probabilistic execution. Observe 
that, since the timed trace distribution of H4 is D’, and since by construction in D’ each occur- 
rence of a start action is followed eventually by the corresponding end action with probability 
1, in HS each occurrence of a start action is followed eventually by the corresponding end 
action with probability 1. Thus, telp can be applied, and ¢-tdistr(telp( H4)) = D, which is a 
contradiction. | 


Lemma 11.5.6 Let C be a distinguishing time-deterministic, cycle-free, separated context for 
two probabilistic timed automata My, and Moz that from any state enables either time-passage 
actions only or discrete actions only. Then there exists a distinguishing time-deterministic, 
cycle-free separated context C’ for My and Mz that from any state enables either time-passage 
actions only or discrete actions only, and such that the transition relation from any state 
enabling discrete actions is at most countably branching. C’ is called a time-deterministic, 
countably-branching, cycle-free separated context. 


Proof. Let D a timed trace distribution of M,||C that is not a timed trace distribution of 
Mz\|C. Consider one of the corresponding probabilistic executions H. Observe that H has at 
most countably many states that enable discrete actions, and that at each state of H there are 
at most countably many transitions of C’ that are scheduled. Thus, in total, only countably 
many discrete transitions of C are used to generate D. Then C’ is C without the useless discrete 
transitions. | 


Lemma 11.5.7 Let C be a distinguishing tame-deterministic, countably-branching, cycle-free 
separated context for two probabilistic timed automata M, and Mz. Then there exists a dis- 
tinguishing cycle-free separated context C’ for M, and My that at each state enabling discrete 
actions either enables two deterministic transitions or a unique probabilistic transition with two 
possible outcomes. C” is called a time-deterministic, binary separated context. 
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Proof. The context C’ is built from C’ in the same way as in the proof of Lemma 7.5.6. The 
constructions shr and shf work as well. The specific procedure is the same as the procedure 
followed in the proof of Lemma 11.5.3. | 


Lemma 11.5.8 Let C be a distinguishing time-deterministic, binary separated context for two 
probabilistic timed automata My, and Mz. Then there exists a distinguishing time-deterministic, 
binary separated context C’ for My and M2 where all the probabilistic transitions have a uniform 
distribution over two states. C’ is called a time-deterministic, balanced separated context. 


Proof. The context C’ is built from C in the same way as in the proof of Lemma 7.5.7. The 
specific procedure is the same as the procedure followed in the proof of Lemma 11.5.3. | 


Lemma 11.5.9 Let C be a distinguishing tame-deterministic, balanced separated context for two 
probabilistic timed automata My, and Mz. Then there exists a distinguishing time-deterministic, 
binary separated context C' for My and Mz with no internal actions and such that for each time 
t each discrete action appears exactly in one edge of the transition tree that leaves from a state 
whose time is t. C’ is called a time-deterministic, total balanced separated context. 


Proof. The context C’ is obtained from C’ by renaming all of its discrete actions so that for 
each time ¢ each edge of the new transition relation leaving from a state whose current time is 
t has its own action. The proof of Lemma 7.5.8 applies. | 


Lemma 11.5.10 Let C be a distinguishing time-deterministic, total balanced separated context 
for two probabilistic timed automata My, and My. Then there exists a distinguishing time- 
deterministic, total, cycle-free separated context C’ for M, and My that from every state en- 
ables one time-passage transition for each timed-action d, two deterministic transitions, and a 
probabilistic transition with a uniform distribution over two choices. C' is called a complete 
context. 


Proof. In this case it is enough to complete C’ by adding all the missing transitions and states. 
If D is a timed trace distribution of M,||C that is not a timed trace distribution of M2||C, then 
it is enough to use on M,||C’ the same scheduler that is used in Mj||C. In fact, since each new 
discrete transition of C’ has a distinct action, none of the new discrete transitions of C’ can be 
used in M2||C’ to generate D, and since each state of C’ is uniquely determined by the timed 
trace of all the executions leading to that state, none of the new time-passage transitions can 
be scheduled (this would affect the resulting timed trace distribution). a 


Lemma 11.5.11 Let C be a distinguishing complete context for two probabilistic tamed au- 
tomata My, and My. Then the timed principal context is a distinguishing context for M, and 
My. 


Proof. The result is achieved in two steps. First the actions of C are renamed so that each 
state enables two deterministic transitions with actions left and right, a probabilistic transition 
with actions pleft and pright, and one transition for each time-passage action d. Call this 
context Cy. Then, by observing that the state of Cy is uniquely determined by the timed trace 
of any timed execution leading to it, all the states of C, are collapsed into a unique one. 
Thus, we need to show only that Cy is a distinguishing context. The proof of Lemma 7.5.10 
applies. | 
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Lemma 11.5.12 Let Cp be a distinguishing context for two probabilistic timed automata My 
and Mz. Then the simple context C of Figure 7-6 augmented with a self-loop time-passage 
transition from state sq for each time-passage action d, where start is an action that does not 
appear in My, and Mo, is a distinguishing context for M, and M3. 


Proof. The proof of Lemma 7.5.11 applies. | 


Proof of Theorem 11.5.1. Let My; Epc; Mz. Then, from Lemma 11.5.12, Mi||Cp Coy 
M2\|Cp. Conversely, let My||Cp Cp: M2||Cp. Then, from Lemmas 11.5.3, 11.5.4, 11.5.5, 
11.5.6, 11.5.7, 11.5.8, 11.5.9, 11.5.10, and 11.5.11, My Epc: Mo. 7 
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Chapter 12 


Hierarchical Verication Timed 
Simulations 


12.1 Introduction 


The simulation method extends to the timed framework almost directly. The main difference 
is that in a timed simulation that abstracts from internal computation we use moves (cf. Sec- 
tion 9.4) rather than weak combined transitions. The kind of results that we prove are a direct 
extension of similar results for the untimed model. In particular, probabilistic timed forward 
simulations are sound for the timed trace distribution precongruence. 


12.2 Probabilistic Timed Simulations 


We start directly with simulation relations that abstract from internal computation; the strong 
relations are essentially the same as for the untimed case. 

For convenience assume that M, and Mp) do not have common states. A probabilistic timed 
bisimulation between two simple probabilistic timed automata M, and Mg is an equivalence 
relation R over states(M,)U states( M2) such that 


1. each start state of My is related to at least one start state of M2, and vice versa; 


2. for each pair of states s; R s2 and each transition s; —+ P, of either M, or Mo, there 
afext(M2) 


exists amove s. ~~ P» of either M, or Moz such that P] =R Po. 
We write My ~p;, Mz whenever ext(M1) = ext( Mz) and there is a probabilistic timed bisimu- 
lation between M, and Mo. 
A probabilistic timed simulation between two simple probabilistic timed automata MM, and 
Mz is a relation RC states(M,) x states( Mz) such that 


1. each start state of Mj, is related to at least one start state of Mo; 


2. for each pair of states s; R sg and each transition s; —+ P, of M;, there exists a move 


atest(Ma) P2 of Mz such that P; Er Po. 
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We write My, Cp; Mz whenever ext(.M1) = ext( M2) and there is a probabilistic timed simulation 
from M, to Mz. We denote the kernel of probabilistic timed simulation by =px. 


It is easy to check that ~p; is an equivalence relation, that Cp; is a preorder relation, and 


that both ~p; and Cp; are preserved by the parallel composition operator. It is also easy to 
verify that a weak probabilistic bisimulation is a probabilistic timed bisimulation and that a 
weak probabilistic simulation is a probabilistic timed bisimulation. 


12.3. Probabilistic Timed Forward Simulations 


A probabilistic timed forward simulation between two simple probabilistic timed automata 
My, Mg is a relation RC states( M1) x Probs(states(M2)) such that 


1. each start state of Ad, is related to at least one Dirac distribution over a start state of 
Md; 
2. for each s R P’, if s “+ Py, then 
(a) for each s’ € 9’ there exists a probability space P, such that s’ afent(Ma) Pet, and 
(b) there exists a probability space Pj of Probs( Probs(states( M2))) satisfying P; Cr Pj, 
such that Yo eq: P'[s'|Ps = Vpeq: Pi[PIP. 


Denote the existence of a probabilistic timed forward simulation from M, to Mz by M, Ers; Mo. 


Proposition 12.3.1 Crs; is preserved by the parallel composition operator. 


Proof. Let M, Crs; Mo, and let R be a probabilistic timed forward simulation from M, to 
My. Let R’ be a relation between states(M,) x states( M3) and Probs( states( M2) x states(M3)), 
defined as follows: 


(31,53) R’ P iff P = P2 © D(s3) for some P2 such that s1 R Po. 


The proof that 7’ satisfies Condition 1 and that Condition 2 is satisfied for each discrete 
transition of M,||Mz is essentially the proof of Proposition 8.5.1. Thus we need to show only 
that Condition 2 is satisfied by time-passage transitions. 

Let (s1, 53) R’ Pz ® D(s3), and let (51, 83) 4, (s1, 55), where s, 4, si, and 83 4, 85. 
From the definition of a probabilistic timed forward simulation, for each s € Q»2 there exists 


a move 8) “+ P, of Mp, and there exists a probability space P3 of Probs( Probs(states(M2))), 
such that 


SS” P.[s]Ps = S> P3[PIP, (12.1) 


and 


D(s,) Er P3. (12.2) 


Moreover, from the definition of a probabilistic timed automaton, there is a trajectory w3 for 
$3 S85. 

For each s € Q2, let O, be a generator for s a, P,. Define a new generator Of as follows: 
for each finite execution fragment a of M2||Ms starting in (s, 53), 
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1. if O,(a[M2) = (s',P), where (s’,P) = 3°; pi(s’, ai, Pi), each (s’,a;,P;) is a transition 
of Mz, and a/ Mz is consistent with ws, i.e., for each prefix a’ of a, Istate(a’)[M3 = 
w3(ltime(a’)), then letting s$ denote Istate(a[ Ms), 


O%(a) = do pil(s 83), aj, Pi ® P:), 


where P} = D(s9) if a; is a discrete action, and P! = D(ws(Itime(a) + a;)) if a; is a 


time-passage action. 
2. otherwise, O(a) = D(6). 


The move generated by each Of is (s, 53) 4 P, ® D(s5). In fact, an execution fragment a 
of M||Mg3 is terminal for Of iff a[M is terminal for O, and Istate(a[M3) = 55, and thus 


Qo: = 1, x D(s3). Moreover, for each a € Qo:, Pos = PON 
Denote P; ® P(s3) by P(s.s,). Then, for each (s, 83) € Q2@ D(s3), we have identified a move 
(8,83) ~ P(s,s,)- These are the spaces of Condition 2.a in the definition of a probabilistic timed 
forward simulation. 
From this point the proof proceeds exactly in the same way as the proof of Proposition 8.5.1. 


No modification of the text is necessary. | 


12.4 The Execution Correspondence Theorem: Timed Ver- 
sion 


The execution correspondence theorem of Chapter 8 extends easily to the timed framework. In 
this section we define the notion of a timed execution correspondence structure, show the timed 
version of the execution correspondence theorem, and, as a consequence, show that probabilistic 
timed forward simulations are transitive. 

The timed execution correspondence theorem is stated in terms of the probabilistic execu- 
tions of a probabilistic timed automaton; however, it is easy to see that the same result can be 
extended to probabilistic timed executions: the execution correspondence theorem talks about 
countably many states of a probabilistic timed execution; all the other points can be described 
by arbitrary trajectories. 


12.4.1 Timed Execution Correspondence Structure 


The definition of a fringe for a probabilistic timed execution is the same as the definition of a 
fringe for a probabilistic execution. For the definition of fringe(H,7) the only difference is in 
the way the length of a state of H is measured, and thus the definition given for probabilistic 
automata is still valid. 

Let R be a probabilistic timed forward simulation from MM, to Mo. A timed execution corre- 
spondence structure via R is a tuple (Hy, H2,m,5'), where H, is a probabilistic execution of My, 
His a probabilistic execution of Mz, mis amapping from natural numbers to fringes of M2, and 
S is a mapping from natural numbers to probability distributions of Probs( Probs(states(H2))), 
such that 
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1. For each i, m(2) < m(i + 1); 
2. For each state gz of He, limiroo Vyea;lar<q Pilg] = PalCal; 
3. Let qm R (Q, F, P) iff for each q € Q, t-trace(q) = t-trace(q,), and either 


(a) qi does not end in 6, each state of 2 does not end in 6, and Istate(q,) R Istate(P), 
or 


(b) q and each state of 2 end in 6 and Istate(é-strip(q,)) R lstate(é-strip(P)). 


Then, for each 7 > 0, m(t) = Urns Psti) [P|P, and fringe(H1,7) Cr S(t). 


4. Let, for each i > 0, each qi € fringe(H1,7), and each q2 € states(H2), Wi(m, @) = 
Yp wig, P)Plg]. If Wi(a,¢5) = 0 for each prefix or extension q5 of q2, then, for each 
extension gj of q such that gq, € fringe(H1,7+ 1), and each prefix or extension gq) of qa, 


Wi41(G %) = 9. 


12.4.2 The Main Theorem 


Theorem 12.4.1 Let M, Ers Mo via the probabilistic timed forward simulation R, and let 
A, be a probabilistic execution of My. Then there exists a probabilistic execution Hz of Mo, a 
mapping m from natural numbers to fringes of Mg, and a mapping S' from natural numbers to 
probability distributions of Probs( Probs(states( Hz))), such that (Hy, H2,m,5') is an execution 
correspondence structure via R. 


Proof. The proof has exactly the same structure as the proof of Theorem 8.6.1. Note that the 
only difference between this theorem and Theorem 8.6.1 is in Condition 3, where we use timed 
traces rather than traces. | 


12.4.3. Transitivity of Probabilistic Timed Forward Simulations 


The timed execution correspondence theorem can be used to show that probabilistic timed 
forward simulations are transitive, i.e., if My LCrs; Mo and My Crs; Mz, then My Crs; Mz. 
The proof of this result follows the same lines as the corresponding proof in the untimed case 
(cf. Section 8.6.4), where combined transitions are replaced by moves and traces are replaced 
by timed traces. We leave the details of the proof to the reader. 


12.5 Soundness for Timed Trace Distributions 


As for the untimed model, the timed execution correspondence theorem can be used to show 
that probabilistic timed forward simulations are sound for the timed trace distribution precon- 
eruence. Since Eps; is a precongruence, it is enough to show that Erg; is sound for the timed 


trace distribution preorder. 


Proposition 12.5.1 Jf M, Crs: Mo, then My Cp; Mo. 
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Proof. Let M, Crs; M2, and let H, be a probabilistic execution of MM, that leads to a timed 
trace distribution D,;. From Lemma 12.4.1, there exists a probabilistic execution Hz of M2 
that corresponds to H, via some mappings m,5. We show that Hy» leads to a timed trace 
distribution D2 that is equivalent to P,. 

Consider a cone C’g of Dj. The cone C’g can be expressed as a union of cones of Py,, and 
thus its measure can be expressed as 


lim S- Pu, [Ca]. (12.3) 


qi €fringe (Hy ,i)|G<t-trace(q1) 


Consider a cone Cg of Dz. The cone C’g can be expressed as a union of cones of Py,, and thus 
its measure can be expressed as 


lim S- Pla: (12.4) 


1 CO 
q2Em(t)|B<t-trace(qo) 


The reason for Expression (12.4) is that at the limit each cone expressing the occurrence of 3 
is captured completely. 
Thus, it is sufficient to show that for each finite G and each 2, 


» Pu, [Ca] = S- Prciyldl- (12.5) 


qi Efringe( Hy ,i)|G<t-trace(q1) q2€m(t)|B<t-trace(q2) 


From this point the proof proceeds exactly as the proof of Proposition 8.7.1. | 
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Chapter 13 


Conclusion 


13.1 Have we Met the Challenge? 


We have developed a model for the description of randomized distributed real-time systems, and 
we have investigated how the new model can be used for the analysis of algorithms. The main 
idea behind the model is to extend labeled transition systems to account for randomization in 
such a way that probabilistic behavior and nondeterministic behavior are clearly distinct. 

We have shown how commonly used informal statements can be formulated in the new 
formalism, and we have shown how such statements can be proved to be correct in a formal 
and rigorous way. In particular, we have developed verification techniques that resemble the 
common ways in which randomized algorithms are analyzed. The main improvement is that 
now we have a collection of results that allow us to determine when a specific argument can be 
used safely. Furthermore, we have shown how to derive upper bounds to the complexity of a 
randomized distributed algorithm using an ordinary time complexity measure as well as more 
abstract complexity measures like “number of rounds in an asynchronous computation”. 

Finally, we have extended several verification techniques that are commonly used within the 
labeled transition system model. We have extended the trace semantics of labeled transition 
systems and several of the existing simulation relations for labeled transition systems. In 
particular, all our preorder relations are compositional and the simulation relations are sound 
for the trace-based semantics. Although we have not presented any example of verification 
using simulations, except for two toy examples based on coin flips, we are confident that in the 
future the method based on simulations will become of practical relevance as it happened for 
ordinary automata. 

Therefore, we can claim that we have met the challenge given by randomization at least 
partially. Surely we understand much more of the problem than before. The fact that we have 
been able to prove new results about randomized algorithms is a positive sign. In particular, 
Aggarwal [Agg94] used successfully the technique presented in this thesis for the verification of 
the randomized self-stabilizing algorithm of Aggarwal and Kutten [AK93], which is not trivial 
at all; during the verification process Aggarwal discovered also a subtle bug in the original 
protocol. In the measure in which the power of a proof method is evaluated based on the bugs 
that such method helps to discover, our methodology has achieved something. Indeed we have 
discovered another bug on one existing algorithm, and the main issue is that we did not have 
to work much to discover such a bug; essentially it was sufficient to try to reformulate the proof 
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of correctness in our framework. 


13.2. The Challenge Continues 


Although we have improved considerably our understanding of randomization in distributed 
computation, what we have discovered looks like the tip of the iceberg. We have addressed 
several problems, and in solving them we have addressed more the basic methodology rather 
than an extensive analysis of all the possible solutions. Therefore, there are several directions for 
further research that can be pursued. Here we suggest some of the most important directions. 


13.2.1 Discrete versus Continuous Distributions 


Throughout this thesis we have assumed that the probability distributions associated with the 
transitions of a probabilistic automaton are discrete. Although such assumption is sufficiently 
general for the study of several randomized algorithms, several other real-time systems are better 
described by using continuous distributions. Examples involve algorithms for transmission of 
data along a common wire, scheduling algorithms for massively parallel machines, and queuing 
systems. Moreover, continuous distributions would be more suitable for the study of randomized 
hybrid systems. 

The extension of the theory to continuous distributions involves nontrivial measure theoret- 
ical problems. In particular it is not the case any more that any union of cones is measurable; 
thus, not even the event that expresses the occurrence of an action or the reachability of a 
state is measurable in general. The events with probability 0 need a more careful treatment 
within the model with continuous distributions. It is likely that some restrictions must be 
imposed to the model to ensure that some minimal set of events is measurable. Examples of 
restricted models with continuous distributions are the automata of Alur, Courcuobetis and 
Dill [ACD91a, ACD91b], where the time that elapses between two transitions is governed by 
an exponential distribution or by a distribution which is non zero in a finite collection of closed 
intervals, and the models of [GHR93, Hil93, BDG94], where the time between the occurrence 
of two actions is assumed to be distributed exponentially. Exponential distributions occur in 
several real systems and are easy to model due to their memoryless structure. However, other 
distributions should be studied. 


13.2.2 Simplified Models 


Within the context of ordinary automata Lynch and Tuttle [LT87] have developed a model of 
I/O automata. The model enforces a distinction between Input actions and Output actions 
within an automaton, and requires that input actions are enabled from every state. Further- 
more, in a parallel composition context each action is required to be the output or internal 
action of at most one process, i.e., each action is under the control of at most one process. 
Based on the Input/Output distinction Lynch and Tuttle can introduce fairness in the model 
in a natural way, and in particular they can use the trace semantics as a meaningful notion of 
implementation. In general the trace semantics is not meaningful as a notion of implementation 
since, for example, it is not sensitive to deadlock. The advantage of the use of traces is that 
traces are easy to deal with. 
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Figure 13-1: Synchronization for probabilistic I/O automata. 


For this reason, it makes sense to study a theory of probabilistic I/O automata as an 
extension of the model of [LT87] and as a restriction of our model. An interesting point of a 
model with I/O distinction is that it is possible to relax the requirement that all the transitions 
of a probabilistic I/O automaton are simple. In particular, only the transitions with input 
actions need to be simple, while all the others can be general. The parallel composition can be 
defined easily since a non-simple transition synchronizes only with simple transitions. Figure 13- 
1 gives an example of synchronization between a transition with three output actions a, b,¢ and 
two transitions of an I/O automaton with just two input actions a,b. A similar observation 
was made also by Wu, Stark and Smolka in [WS5S94]. 

A restricted timed model with I/O distinction is introduced by Merrit, Modugno and Tuttle 
[MMT91]. In particular timing constraints can be described only by giving upper and lower 
bounds to the time it takes for a process to perform the next transition whenever it is ready 
to do so. MMT automata turned out to be sufficient for the modeling of several distributed 
systems, and in particular, due to their simple structure, made the analysis simpler than by 
using the full automaton model. Once again, a study of the probabilistic version of the MMT 
model would be useful. The proofs that we have illustrated in Chapter 12 could be carried out 
in the probabilistic MMT model as well. 

Finally, the analysis of a system can be simplified by studying time-deterministic probabilis- 
tic timed automata, i.e., probabilistic timed automata such that from each state s and each time 
d there is at most one state reachable from s in time d. In fact, if a system is time-deterministic, 
then the end points of a time-passage transition determine completely the trajectory that is 
spanned. Therefore, trajectories could be removed also from the direct analysis of randomized 
timed algorithms. It turns out that most of the times an algorithm can be described as a 
time-deterministic probabilistic automaton. Probabilistic MMT automata are an example of 
time-deterministic probabilistic automata. 


13.2.3. Beyond Simple Probabilistic Automata 


The study of parallel composition and of the simulation relations of this thesis is done within 
the context of simple probabilistic automata. The main problem is that we did not find any 
reasonable definition of parallel composition for general probabilistic automata that is consistent 
with our synchronization style. We have just observed that in the presence of an Input/Output 
distinction it is possible to relax the simplicity condition and yet obtain a meaningful notion 
of parallel composition. It would be interesting to investigate other mechanisms that give a 
meaning to general probabilistic automata and yet work as we expect in the simple case. 
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13.2.4 Completeness of the Simulation Method 


We have provided several simulation and bisimulation relations for probabilistic automata and 
probabilistic timed automata, and we have shown that they are sound for the trace distribution 
precongruence and the timed trace distribution precongruence, respectively. However, we have 
not shown any completeness result for probabilistic forward simulations and probabilistic for- 
ward timed simulations. In [LV93a, LV95] it is shown that forward simulations together with 
another kind of simulations called backward simulations are sound and complete for the trace 
preorder. Are probabilistic forward simulations complete for the trace distribution preorder? 
If not, is there an equivalent of backward simulations that can lead to completeness? 


13.2.5 Testing Probabilistic Automata 


We have presented the trace distribution semantics as an example of a semantics based on 
abstract observations. Another widely known semantics for ordinary automata is the failure 
semantics of Brookes, Hoare and Roscoe [BHR84], which in turn is connected to the testing 
preorders of De Nicola and Hennessy [DH84]. Similarly to the trace distribution semantics, 
it should be possible to extend the failure semantics to the probabilistic framework and find 
a sufficiently powerful context to distinguish probabilistic automata that are not in the corre- 
sponding precongruence relation. Possibly, a related theory of testing in the style of [DH84] 
should be defined. It is very likely that the new testing preorders will be similar to those 
of Yi and Larsen [YL92]. Other theories of testing for probabilistic automata are studied in 
[Chr90b, Chr90a, CSZ92, YCDS94] and are explained in Section 2.2. 


13.2.6 Liveness in Probabilistic Automata 


In the extension of the notion of an execution of an automaton we have obtained a parallelism 
between the theory of ordinary automata and the theory of probabilistic automata. In this 
parallelism also the notion of liveness has found its place, although we have not addressed the 
issue in this thesis. In ongoing research we have given a simple definition of a live probabilistic 
automaton as a pair (M,L) where L is an arbitrary subset of the probabilistic executions of M, 
and we have shown that the live trace distribution precongruence can be defined easily and can 
be characterized by a live principal context, which is essentially the principal context paired 
with the set of its probabilistic executions. However, lot of work remains to be done within the 
theory of liveness. 

First of all it would be useful to study how the definition of safety and liveness properties 
of Alpern and Schneider [AS85] extends to the probabilistic framework and what consequences 
such extension has. Furthermore, the use of the live trace preorder within ordinary automata 
makes sense as a notion of implementation in the presence of I/O distinction and of a property 
called receptiveness or environment-freedom |Dil88, AL93, GSSL94]. It would be useful to 
study the theory of receptiveness of [Dil88, AL93] and of environment-freedom of [GSSL94] 
in the context of randomization. In this case, differently from [GSSL94], the environment is 
expressed by a function rather than by a sequence of actions. However, non-trivial problems 
arise in imposing restrictions to the behavior of the environment. 
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13.2.7 Temporal Logics for Probabilistic Systems 


In the chapters on direct analysis we have identified a collection of probabilistic statements 
that are useful for the analysis of algorithms. However, there are several other statements that 
can be of interest. It would be desirable to find a probabilistic temporal logic that expresses 
as many properties as possible. The probabilistic modal logic of [LS89] is a direct extension of 
the modal logic of Hennessy and Milner [HM85] for reactive processes, but it is not sufficiently 
powerful to deal with nondeterminism; similarly, the extended probabilistic logic of [LS92] is not 
sufficiently powerful. The Probabilistic Computation Tree Logic of [HJ89, Han94] captures more 
the consequences of the interplay between probability and nondeterminism; in [SL94] PCTL is 
generalized also to probabilistic systems with internal actions (WPCTL). However, there are 
still properties that are useful and do not seem to be expressible in WPCTL. Specifically, we 
do not know how to express a property of the kind “after something has happened, no matter 
where I am, something else will happen with probability at least p”. Is there something missing 
in WPCTL? What would be a more appropriate temporal logic? 

Another issue is the relationship between the simulation method and temporal logic. That 
is, if a probabilistic automaton implements another probabilistic automaton according to some 
implementation relation (e.g., trace distribution precongruence, probabilistic simulation, proba- 
bilistic forward simulation, etc.), what can we say about the implementation? What properties 
of the specification are satisfied by the implementation? More generally, given a probabilis- 
tic temporal logic and a preorder relation, what fragment of the logic is preserved by the 
preorder relation? Somehow it is implicit that whenever we use some preorder relation as a 
notion of implementation we are interested only in the properties that are preserved by such 
relation; however, we need to know what are those properties. In [SL95] we have stated that 
weak probabilistic simulation preserve a large fragment of WPCTL and that weak probabilistic 
bisimulations preserve WPCTL. The results of [SL95] can be proved easily given the results of 
this thesis. However, more work in this direction is necessary. In particular, some completeness 
results would be useful. 


13.2.8 More Algorithms to Verify 


In this thesis we have illustrated our direct verification technique by proving the correctness 
of the randomized dining philosophers algorithm of Lehmann and Rabin [LR81] and of the 
randomized agreement protocol of Ben-Or [BO83]. In [Agg94] Aggarwal uses our model to verify 
the correctness of the self-stabilizing minimum weight spanning tree randomized algorithm of 
Aggarwal and Kutten [AK93]. However, the technique should be tested against many other 
algorithms. We are currently investigating the agreement protocol of Aspnes and Herlihy [AH90] 
and the randomized mutual exclusion algorithm of Pnueli and Zuck [PZ86]. Based on the little 
experience that we have gained, we can say that the model provides us with a systematic way 
of analyzing those algorithms, and in particular it provides us with a simple methodology to 
identify the critical points of an algorithm. 

It is very likely that new coin lemmas need to be developed together with other techniques 
for the actual computation of the probability of an event. A technique that needs further 
development is the partition technique of Section 6.7. The analysis of other algorithms should 
make clear what other techniques are necessary. Also, playing with the toy resource allocation 
protocol of Chapter 5 can be very instructive. Although the protocol is simple, its analysis 
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highlights several of the issues that arise in randomized distributed computation. 

It is also plausible, as it happened for non-probabilistic distributed algorithms, that some 
complex protocols can be verified more easily by using the simulation method. Finding those 
algorithms would be an optimal way to test the hierarchical verification method and possibly 
to improve it. 


13.2.9 Automatic Verification of Randomized Systems 


Formal verification usually involves two levels of analysis. First, an algorithm is analyzed at 
a high level by using the intuition that designers have of their own algorithm; then, a more 
detailed verification of the high level claims is carried out in order to guarantee correctness. 
The low level analysis is very tedious and involves checking a whole lot of uninteresting details. 
On the other hand, several times the low level analysis is the only way to discover flaws in the 
intuitions about an algorithm. 

Fortunately, the low level analysis is amenable to automatic verification, although the re- 
search in this area is still in progress. Model checking [EC82, CES83] is certainly a useful 
technique; in [SGG*93] it is shown how a theorem prover can be used to help in the verification 
of a protocol using simulations; in [P595] we have investigated how a randomized algorithm 
can be verified mechanically once the high level proof is formulated. However, there is still a 
lot of work that needs to be done. It would be interesting to study how model checking and 
theorem proving could be integrated to automatize part of the verification of an algorithm. 


13.3. The Conclusion’s Conclusion 


To say what we have done in one sentence, we have provided a new way of reasoning about 
randomized systems that integrates both the theoretical aspects of modeling and the basic 
requirements for usage in practice. From the modeling point of view we have distinguished be- 
tween nondeterminism and probability explicitly and we have extended the main semantics that 
are available within the labeled transition systems model; from the point of view of verification 
we have formalized some of the common informal arguments about randomized algorithms and 
we have provided guidelines to determine whether an argument can be used safely. Further- 
more, we have provided a systematic way to analyze the complexity of randomized algorithms. 
All our results are compatible with previous work. 

As we have seen in the previous section, there are still many open problems in this area. 
Here we hope to have stimulated the curiosity of the reader to go much further. Needless to 
say that for us (me) working on this project was a continuous discovery. 
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Istate() Last state of. 39 
frag() Execution fragments of. 39 
exec( ) Executions of. 39 
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Transition suffixing operator. 

Trace. 

Traces of. 

Trace preorder. 

Parallel composition operator. 

Probabilistic automaton. 

Termination or deadlock symbol. 
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GFIRST(S,E)() Coin event: first occurrence of an action among many with several 122 


outcomes. 
GCOIN(S,E)() General coin event. 125 
D Trace distribution. 138 
tdistr() Trace distribution of. 138 
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itdistr() Internal trace distribution of. 139 
itdistrs() Internal trace distributions of. 139 
Cc Trace distribution preorder. 141 
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C'p Principal context, timed principal context. 145 
ptdistrs() Principal trace distributions of. 146 
Cc Lifting of a relation to probability spaces. 168 
~ Existence of a strong bisimulation. 169 
Egg Existence of a strong simulation. 169 
~p Existence of a strong probabilistic bisimulation. 171 
Lsps Existence of a strong probabilistic simulation. 171 
=p Existence of a weak probabilistic bisimulation. 172 
Cwes Existence of a weak probabilistic simulation. 172 
Crs Existence of a probabilistic forward simulation. 174 
vis() Visible actions of. 196 
Ww Trajectory. 197 
ltime() Last time of. 197 
t-frag() Timed execution fragments of. 199 
t-exec( ) Timed executions of. 199 
t-execs( ) Extended timed executions of. 199 
te-frag() Time-enriched execution fragments of. 201 
te-prfrag() Probabilistic time-enriched execution fragments of. 202 
te-prexec() Probabilistic time-enriched executions of. 202 
sample() Function that applied to a probabilistic time-enriched execution 209 


HT of a probabilistic timed automaton M returns a probabilistic 
execution H’ of M that samples H. 

t-sample() Function that applied to a probabilistic time-enriched execution 211 
fragment H of a probabilistic timed automaton M returns a prob- 
abilistic timed execution fragment H’ of M that t-samples H. 
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~~ Move. 217 
Ev, Advs[€] Worst expected time for success of the event schema e starting from 227 
a state of U under the action of an adversary from Advs. 
seq() Sequence of a timed sequence pair. 243 
tsp() Timed sequence pairs over some given set. 243 
t-trace() Timed trace of. 244 
t-tdistr() Timed trace distribution of. 246 
t-tdistrs() Timed trace distributions of. 247 
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Lpct 
pt-tdistrs() 
~Pt 

Cpt 

Crst 


Timed trace distribution preorder. 

Timed trace distribution precongruence. 

Principal timed trace distributions of. 

Existence of a probabilistic timed bisimulation. 
Existence of a probabilistic timed simulation. 
Existence of a probabilistic timed forward simulation. 
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abstract complexity, 238 
action, 37 
discrete, 196 
hiding operator, 73 
renaming operator, 72 
restriction, 139, 249 
signature, 37 
time-passage, 196 
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adversary, 19, 75, 79, 224 
deterministic, 79, 80, 224 
oblivious, 91 
schema, 80 
with partial on-line information, 79 
alternating model, 28 
automaton, 37 
fully probabilistic, 47 
probabilistic, 18, 46 
probabilistic Input/Output, 265 
probabilistic MMT, 265 
probabilistic semi-timed, 196 
probabilistic timed, 196 
simple probabilistic, 47 
timed, 195 


behavioral semantics, 135 
bisimulation 
probabilistic timed, 257 
strong, 169 
strong probabilistic, 171 
weak probabilistic, 172 


coin 
event, 103 
lemma, 103, 104 
coin lemma, 19 
compatibility, 41, 61 
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of two event schemas, 83 
of two executions, 39 
of two time-enriched executions, 201 
of two timed executions, 199 
of two trajectories, 199 
conditional 
event, 36 
of a probabilistic execution, 57 
of a probabilistic time-enriched execu- 
tion, 203 
of a probabilistic timed execution, 207 
probability space, 36 


Dirac distribution, 37 


event, 34 
schema, 82, 224 
execution, 39 
admissible timed, 198 
extended, 50 
finite timed, 198 
probabilistic, 19, 49 
probabilistic time-enriched, 202 
probabilistic timed, 200, 205 
time-enriched, 201 
timed, 198 
timed extended, 199 
Zeno timed, 198 
execution correspondence structure, 177 
timed, 259 
execution-based 
adversary schema, 79, 91 
event schema, 79, 83 
expected time of success, 227 
expected value of a random variable, 36 
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probabilistic execution, 55 
probabilistic time-enriched execution, 203 
probabilistic timed execution, 206 
finite-history insensitivity, 86 
finitely satisfiable 
event, 53 
event schema, 82 


generative process, 23, 25 
generator 
of a o-field, 33 


of a weak transition, 60 


internal trace, 139 
distribution, 139 
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measurable 
function, 34 
set, 33 
space, 33 
measure induced by a function, 35 
measure space, 34 
complete, 34 
discrete, 34 
model checking, 17, 30, 31 
move, 217 


oblivious relation, 92 
observation, 135 
observational semantics, 135 


parallel composition 
of automata, 41 
of simple probabilistic automata, 61 
of simple timed probabilistic automata, 
218 
partial on-line information, 92 
partition technique, 20, 132 
patient 
construction, 197 
point of extension, 56 
point of satisfaction, 83 
precongruence, 20, 136 
timed trace distribution, 249 
trace distribution, 20, 137, 143 
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prefix 
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of a probabilistic time-enriched execu- 
tion, 203 
of a probabilistic timed execution, 206 
of a time-enriched execution, 201 
of a timed execution, 199 
of a trace distribution, 139 
of an execution, 39 
preorder 
timed trace distribution, 249 
trace distribution, 20, 137, 141 
principal 
context, 20, 137, 145 
timed context, 21, 243, 250 
timed trace distribution, 250 
trace distribution, 20, 137, 146 
probabilistic statement, 19, 84 
probability 
distribution, 34 
measure, 34 
space, 34 
progress statement, 19, 85 
timed, 21, 223, 226 
projection 
of a probabilistic execution, 62, 65 
of a probabilistic time-enriched execu- 
tion, 218 
of a probabilistic timed execution, 218 
of an execution, 41 


qualitative analysis, 29 
quantitative analysis, 29 


random variable, 36 
reachable state, 39, 60 
reactive process, 23, 24 


sample space, 34 

scheduler, 79 

o-additivity, 34 

o-field, 33 

simulation 
method, 137, 167 
probabilistic forward, 20, 174 
probabilistic timed, 257 


probabilistic timed forward, 258 
strong, 169 
strong probabilistic, 171 
weak probabilistic, 172 
stratified process, 24, 25 
substitutivity, 136 
suffix 
of a probabilistic execution, 57 
of a probabilistic time-enriched execu- 
tion, 203 
of a probabilistic timed execution, 207 
of a time-enriched execution, 201 
of a timed execution, 199 
of an execution, 39 


terminal state, 60 
time deadlock, 199 
timed sequence, 243 
timed sequence pair, 243 
trace 
distribution, 20, 137, 138 
of an execution, 40 
timed, 21, 243, 244 
timed distribution, 243, 246 
trajectory, 195, 197 
axioms, 195, 197 
transition, 37 
action restricted, 64 
combined, 47 
prefixing, 52 
relation, 37 
suffixing, 52 
time-enriched, 202 
timed, 205 
weak, 38, 58 
weak combined, 59 


uniform distribution, 37 
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